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ABSTRACT 


The  importance  of  information  superiority  has  been  emphasized  as  a  critical 
capability  that  future  joint  forces  must  be  able  to  achieve.  No  longer  simply  a  future 
concept,  it  is  being  officially  defined  and  incorporated  in  doctrinal  publications  like  Joint 
Publication  3-13,  Information  Operations.  Unfortunately,  our  ability  to  effectively 
measure  its  contribution  relative  to  other  battlefield  systems  remains  limited.  This 
research  focuses  on  exploring  the  limits  of  the  contributions  that  information  superiority 
can  make,  examining  the  sensitivity  of  information  superiority  to  varying  information 
quality  and  comparing  those  contributions  with  other  contributing  factors  to  battlefield 
results.  Furthermore,  an  effort  is  made  to  identify  some  of  the  risks  associated  with  using 
information  superiority  as  a  force  multiplier.  A  simple  decision  model  was  developed 
based  on  the  concepts  of  a  two-person  zero  sum  game  to  explore  these  questions.  In  the 
model,  one  side  is  provided  varying  degrees  of  an  information  advantage,  while  also 
varying  degrees  of  infonnation  quality  to  the  infonnation  advantage.  Additionally,  a 
variety  of  scenarios  were  considered  involving  varied  levels  of  opposing  side  force 
levels.  Experimental  design  techniques  were  employed  to  efficiently  explore  the  model 
output  space  while  allowing  for  sufficient  replications  of  the  model  at  each  design  point 
in  order  to  provide  a  sufficient  data  set  for  analysis. 
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EXECUTIVE  SUMMARY 


Information  superiority  is  a  leading  concept  driving  joint  future  force 
development.  Proponents  view  it  as  a  force  multiplier;  given  forces  of  equal  size  and 
ability,  the  one  that  possesses  infonnation  superiority  can  achieve  superior  results  to  that 
of  the  other.  Research  suggests  that  this  is,  in  fact,  the  case.  Yet,  what  are  the  risks 
associated  with  units  relying  on  information  superiority?  How  can  we  measure  the 
degree  of  superiority  that  an  infonnation  advantage  provides?  How  much  is  enough?  In 
a  world  constrained  by  budgets,  these  are  important  questions  to  be  answered  so  that  a 
proper  balance  can  be  made  between  equipment  meant  to  destroy  our  adversaries  and 
equipment  that  facilitates  information  superiority.  It  has  been  aptly  pointed  out  by 
General  Howell  M.  Estes  III,  United  States  Air  Force,  a  former  commander  of  Space 
Command  that,  “. .  .you  can’t  take  out  an  enemy  tank  with  just  information.”1 

Unfortunately,  attacking  this  type  of  problem  has  proven  difficult  for  the 
Operations  Research  (OR)  community.  Alan  Washburn,  Professor  Emeritus  at  the 
Naval  Postgraduate  School,  views  the  situation  as  somewhat  dire.  He  says, 

There  is  a  crisis  for  military  OR,  centered  on  the  role  of  information  on  the 
battlefield.  It  is  clear  to  military  professionals  that  information  is  becoming 
increasingly  important,  but  unfortunately  the  OR  profession’s  ability  to  measure 
its  contribution  is  still  primitive.2 

This  present  work  will  not  attempt  to  make  a  breakthrough  in  these  “primitive”  measures, 
but,  instead,  will  try  to  creatively  apply  methods  on  hand  to  continue  to  explore  the  value 
of  information  superiority  in  addressing  the  following  research  questions: 

•  How  do  varying  degrees  of  information  superiority  affect  battlefield 
outcomes? 

•  How  sensitive  are  these  outcomes  to  the  quality  of  information  used  to 
obtain  infonnation  superiority? 


1  Alan  R.  Washburn,  “Bits,  Bangs,  or  Bucks?  The  Coming  Information  Crisis,”  PHALANX,  Vol.  34, 
No.  3  (Part  I),  2001,  p.  6. 

2  Ibid.,  p.  6. 
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•  What  is  the  cost  savings  in  terms  of  level  of  forces  that  information 
superiority  provides  (i.e.,  given  a  certain  level  of  success  achieved  through 
information  superiority,  if  information  superiority  is  taken  away,  what 
increase  in  force  size  is  required  to  achieve  similar  results)? 

The  title  of  this  paper  is  meant  to  convey  the  link  that  exists  between  information 
superiority  and  a  decision  maker.  Information  superiority  is  defined  as  “the  operational 
advantage  gained  by  the  ability  to  collect,  process,  and  disseminate  an  uninterrupted  flow 
of  information  while  exploiting  or  denying  an  adversary’s  ability  to  do  the  same.”3 
However,  this  advantage  is  turned  into  improved  battlefield  success  through  the 
decision-making  process.  Therefore,  in  this  study  we  employ  a  simple  decision  model 
based  largely  on  the  concepts  of  game  theory,  specifically  a  two-person  zero  sum 
(TPZS)  game. 

TPSZ  games  are  an  excellent  abstraction  of  military  conflict  since  they  involve 
two  opposing  sides,  each  of  whom  must  decide  between  an  array  of  strategies  in  order  to 
achieve  the  most  desirable  result.  Warfare,  in  its  most  basic  form,  can  be  thought  of  as  a 
series  of  decisions  that  are  executed  in  the  form  of  strategy  and  tactics  between  opposing 
forces,  where  each  side  desires  a  higher  payoff.  In  the  case  of  the  TPZS  game  where 
both  sides  have  no  information  as  to  how  the  other  will  behave,  the  optimal  action  for 
each  side  is  to  play  their  respective  strategies  with  the  proportion  that  guarantees  the 
maximum  expected  value,  regardless  of  the  strategy  the  adversary  selects.  This  is  known 
as  the  optimal  mixed  strategy.  In  this  situation,  the  opposing  sides  have  information 
parity — neither  holding  an  advantage  over  the  other.  This  situation  serves  as  the  baseline 
of  comparison  to  situations  where  one  side  holds  an  information  advantage. 

In  order  to  incorporate  the  information  advantage  concept,  the  TPZS  game  used  in 
this  study  is  a  larger  decision  model  that  maintains  the  principles  of  the  TPZS  game.  In 
this  larger  decision  model,  one  side  is  consistently  provided  varying  degrees  of 
information  superiority  to  explore  how  the  increased  information  advantage  impacted  the 
expected  payoff.  The  difference  between  this  expected  payoff  and  the  baseline  expected 
payoff  is  measured  as  the  value  of  information  superiority  for  the  given  circumstances  of 

3  Joint  Chiefs  of  Staff,  Joint  Publication  3-13,  “Information  Operations,”  Washington  D.C.,  February 
2006.  p.  1-5. 
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a  specific  game.  Additionally,  the  quality  of  the  information  is  varied,  incorporating 
model  parameters  for  timeliness  and  accuracy  of  information. 

Experimental  results  have  suggested  the  following: 

•  The  value  of  information  superiority  is  not  uniform,  but  is  strongly 
influenced  by  force  ratio  and  force  size. 

•  Information  superiority  can  be  increased  to  a  point  that  it  returns  little  or 
no  value. 

•  Decreasing  information  quality  degrades  the  value  of  information 
superiority  uniformly,  with  no  single  drop-off  point. 

•  While  there  are  risks  to  relying  on  information  superiority  in  terms  of 
potential  decreases  in  battlefield  performance,  there  is  a  much  greater 
potential  for  increased  battlefield  performance. 

It  appears  that  nonlinear  relationships  exist  between  the  value  of  information 
superiority  and  both  force  ratio  and  force  size.  There  is  an  initial  minimal  force 
requirement  before  information  has  much  value.  Additionally,  one  force  size  reaches  a 
certain  point,  the  value  of  information  superiority  begins  to  decline  sharply. 

This  study  is  meant  to  continue  to  contribute  to  a  critical  area  of  operations 
research  where  much  work  remains  to  be  done.  While  not  revolutionizing  the  way  we 
measure  the  contribution  of  information  to  war  fighting,  it  demonstrates  a  practical 
manner  in  which  to  explore  the  problem  and  develop  useful  insights.  Additionally,  by 
building  on  some  of  the  research  that  has  gone  before,  this  study  should  help  illuminate 
future  areas  of  similar  interest. 
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I.  INTRODUCTION 


A.  STUDY  MOTIVATION  AND  THESIS 

Joint  Vision  2020  states  that,  “The  continued  development  and  proliferation  of 
information  technologies  will  substantially  change  the  conduct  of  military  operations.”4 
As  part  of  this  change,  the  concept  of  infonnation  superiority  has  been  identified  as  a  key 
enabler  to  future  U.S.  success  on  the  battlefield.  This  concept  has  now  been  codified  in 
joint  doctrine  such  as  JP  3-13  Information  Operations,  where  it  states,  “To  succeed  it  is 
necessary  for  US  forces  to  gain  and  maintain  information  superiority.”5  This  publication 
goes  on  to  define  information  superiority  as  “the  operational  advantage  gained  by  the 
ability  to  collect,  process,  and  disseminate  an  uninterrupted  flow  of  information  while 
exploiting  or  denying  an  adversary’s  ability  to  do  the  same.”6  With  these  things  in  mind, 
it  is  safe  to  say  that  the  U.S.  military  is  seeking  to  take  advantage  of  these  changes  in 
information  technology  to  achieve  a  sustainable  operational  advantage  on  the  battlefield 
in  the  same  way  that  artillery  with  a  range  of  15  km  has  a  sustainable  operational 
advantage  over  artillery  with  a  range  of  only  10  km.  Part  of  the  promise  of  these  changes 
is  revealed  in  Joint  Vision  2010  (JV  2010),  where  it  is  envisioned  that  “we  should  be  able 
to  change  how  we  conduct  the  most  intense  joint  operations.”7  JV2010  goes  on  to  posit: 
“Instead  of  relying  on  massed  forces  and  sequential  operations,  we  will  achieve  massed 
effects  in  other  ways”8,  thus  avoiding  “risky  massing  of  people  and  equipment.”9  But 
what  are  the  risks  associated  with  smaller,  more  dispersed  units  relying  on  infonnation 
superiority?  And  how  can  we  measure  what  the  contribution  of  information  superiority 

4  Joint  Chiefs  of  Staff,  Joint  Vision  2020,  Joint  Staff  Pentagon,  Washington,  D.C. 
http://www.dtic.mil/iointvision/ivpub2.htm,  accessed  December  2007.  pp.  3-4 

5  Joint  Chiefs  of  Staff,  Joint  Publication  3-13,  “Information  Operations,”  Washington  D.C., 

February  2006.  p.ix. 

6  Ibid.,  p.  1-5. 

7  Joint  Chiefs  of  Staff,  Joint  Vision  2010,  Joint  Staff  Pentagon,  Washington,  D.C. 
http://www.dtic.mil/iv20 1 0/iv20 1 0.pdf,  accessed  December  2007.  p.  17. 

8  Ibid.,  p.  17. 

9  Ibid.,  p.  17. 
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will  be  to  the  “massed  effects”  that  we  desire?  It  has  been  aptly  pointed  out  by 
General  Howell  M.  Estes  III,  United  States  Air  Force,  a  former  commander  of 
Space  Command,  that  “You  can’t  take  out  an  enemy  tank  with  just  infonnation.  We  need 
to  strike  a  balance  between  ‘shooters’  and  ‘information  systems’  if  we’re  going  to  be 
successful  in  the  future.”10 

Alan  Washburn,  Professor  Emeritus  at  the  Naval  Postgraduate  School,  has 
pointed  out  that  the  operations  research  community  is  not  well  positioned  to  address  the 
issues  related  to  the  balance  between  “shooter”  and  “infonnation  systems”  because 
effective  ways  for  measuring  the  contribution  of  information  on  the  battlefield,  in 
general,  are  lacking.  He  says, 

There  is  a  crisis  for  military  OR,  centered  on  the  role  of  infonnation  on  the 
battlefield.  It  is  clear  to  military  professionals  that  information  is 
becoming  increasingly  important,  but  unfortunately  the  OR  profession’s 
ability  to  measure  its  contribution  is  still  primitive.11 

Professor  Washburn  remains  pessimistic  about  our  ability  to  attack  the  problem,  but 
urges  creative  use  of  the  tools  at  hand.  His  article  provides  a  useful  summary  of  available 
tools  that  presents  their  relative  weaknesses  and  limitations. 

Thus,  this  study  takes  aim  at  exploring  what  is  clearly  an  important  research  area 
by  creatively  applying  the  tools  at  hand  to  look  for  insights  into  the  following  questions: 


•  How  do  varying  degrees  of  information  superiority  affect  battlefield 
outcomes? 

•  How  sensitive  are  these  outcomes  to  the  quality  of  information  used  to 
obtain  infonnation  superiority? 

•  What  is  the  cost  savings  in  terms  of  level  of  forces  that  information 
superiority  provides  (i.e.,  given  a  certain  level  of  success  achieved  through 
information  superiority,  if  information  superiority  is  taken  away,  what 
increase  in  force  size  is  required  to  achieve  similar  results)? 

The  first  question  addresses  issues  related  to  how  much  information  superiority  is 
enough.  Of  course,  we  expect  that  an  increase  in  information  superiority  will  result  in 


10  Alan  R.  Washburn,  “Bits,  Bangs,  or  Bucks?  The  Coming  Information  Crisis,”  PHALANX ,  Vol.  34, 
No.  3  (Part  I),  2001.  p.  6. 

11  Ibid.,  p.  6. 
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better  battlefield  outcomes,  but  is  the  relationship  linear  and  how  steep  is  the  slope?  The 
second  question  examines  the  sensitivity  of  the  effects  of  information  superiority  when 
the  information  being  used  to  develop  information  superiority  has  varying  degrees  of 
quality.  The  quality  of  information  is  defined  as  the  information’s  accuracy,  relevance, 
timeliness,  usability,  completeness,  brevity,  and  security.12  For  this  study,  we  will  vary 
information  quality  by  varying  the  accuracy  and  timeliness  characteristics  and  holding 
the  remaining  factors  at  steady  positive  levels.  There  is  research  that  has  examined  the 
effects  of  information  quality  and  information  superiority,13  but  not  much  that  has 
addressed  both  simultaneously,  as  this  work  is  attempting  to  do.  Finally,  the  third 
question  is  looking  for  a  comparative  analysis  between  force  size  and  the  level  of 
information  superiority  possessed.  An  example  comparison  would  be,  given  a  force  of 
specific  size  with  a  certain  level  of  information  superiority  that  can  achieve  some  level  of 
success  in  a  given  scenario,  if  we  create  another  force  without  any  information 
superiority,  how  large  will  it  have  to  be  to  achieve  an  equal  level  of  success  or  chance  of 
success?  This  is  the  aim  of  this  present  work. 

B.  THE  GENERAL  VALUE  OF  INFORMATION 

Information  superiority  provides  the  joint  force  a  competitive  advantage 
only  when  it  is  effectively  translated  into  superior  knowledge  and 
decisions.  The  joint  force  must  be  able  to  take  advantage  of  superior 
information  converted  to  superior  knowledge  to  achieve  ‘decision 
superiority’ — better  decisions  arrived  at  and  implemented  faster  than  an 
opponent  can  react .  .  .  ,14 

Warfare,  in  its  most  basic  form,  can  be  thought  of  as  a  series  of  decisions  that  are 
executed  in  the  fonn  of  strategy  and  tactics  between  opposing  forces.  Viewed  in  this 
light,  it  is  clear  that  information  serves  a  decision  maker  and  its  value  is  related  to  the 


12  Joint  Chiefs  of  Staff,  Joint  Publication  3-13,  “Information  Operations,”  Washington  D.C., 

February  2006.  p.  1-3. 

13  Gary  A.  McCintosh,  ”  Information  Superiority  and  Game  Theory:  The  Value  of  Varying  Levels  of 
Information,”  Master’s  Thesis,  Naval  Postgraduate  School,  Monterey,  CA,  March  2002,  is  a  specific 
expansion  on  Bracken  and  Darilek’s  (1998)  work.  Another  interesting  study  on  information  quality  is 
McGunnigle,  J.  Jr.,  “An  Exploratory  Analysis  of  the  Military  Value  of  Information  and  Force,”  Master’s 
Thesis,  Naval  Postgraduate  School,  Monterey,  CA,  December  1999. 

14  Joint  Chiefs  of  Staff,  Joint  Vision  2020,  Joint  Staff  Pentagon,  Washington,  D.C., 
http://www.dtic.mil/iointvision/ivpub2.htm,  accessed  December  2007.  p.  1 1. 
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degree  in  which  it  can  potentially  improve  a  certain  decision  with  the  ultimate  aim  of 
improving  action  on  the  battlefield.  As  Washburn  points  out,  “Information  is  of  no  value 
unless  there  is  an  uncertain  decision  maker.”15  From  this,  we  conclude  that  infonnation 
superiority  is  not  an  end  in  and  of  itself.  What  is  really  desired  is  decision  superiority, 
better  and  faster  decisions  relative  to  that  of  the  adversary. 

These  ideas  were  captured  very  succinctly  back  in  the  1970s  by  Colonel  John 
Boyd,  United  States  Air  Force.  Boyd’s  decision-making  model,  known  as  the  Observe, 
Orient,  Decide,  and  Act  (OODA)  Loop,  was  developed  as  a  model  for  air-to-air  combat, 
but  is  now  generally  applied  to  all  forms  of  decision  making.  The  first  two  steps 
(Observe  and  Orient)  can  be  considered  as  the  information-processing  steps,  where 
information  is  collected,  organized,  and  interpolated.  The  second  two  steps  (Decide  and 
Act)  can  be  considered  as  the  steps  where  the  information  is  utilized  and  turned  into 
actions  on  the  battlefield.  It  is  a  very  useful  model  for  visualizing  the  basic  dynamic  of  a 
decision  maker.  Of  course,  decision  makers  do  not  always  make  good  decisions.  In  fact, 
sometimes  they  make  very  poor  decisions.  Assuming  we  have  an  individual  in  a  position 
to  make  a  decision,  what  are  some  potential  sources  of  error  or  improvements  that  enter 
into  the  decision-making  process  that  could  effect  the  actual  decision  made?  To  answer 
this,  we  look  to  a  more  complex  model  called  the  Dynamic  Model  of  Situated  Cognition 
(DMSC).16 

The  DMSC  is  a  model  for  describing  the  development  of  a  decision  maker’s 
situational  awareness.  The  result  of  the  information-processing  steps  (Observe  and 
Orient)  is  what  is  generally  known  as  situational  awareness,17  which  is  simply  a  term  to 
describe  an  individual’s  comprehension  of  the  events  going  on  around  them.  It  has  no 

15  Alan  R.  Washburn,  “Bits,  Bangs,  or  Bucks?  The  Coming  Information  Crisis,”  PHALANX,  Vol.  34, 
No.  3  (Part  I),  2001.  p.  6. 

lf)  Nita  L.  Miller  and  Lawrence  G.  Shattuck,  “A  Process  Model  of  Situated  Cognition  in  Military 
Command  and  Control,”  Proceedings  of  the  2004  Command  and  Control  Research  and  Technology 
Symposium,  San  Diego,  CA. 

17  Definitions  for  situational  awareness  are  numerous.  The  Marine  Corps  definition  may  be  the  most 
concise  and  can  be  found  in  MCDP  2,  Intelligence :  “. .  .a  keen  understanding  of  the  essential  factors  which 
make  each  condition  unique — rather  than  on  preconceived  schemes  or  techniques.”,  but  this  hardly 
qualifies  it  as  the  most  used  or  understood.  The  journal  Human  Factors,  Volume  37,  Number  1,  March 
1995,  is  completely  dedicated  to  the  discussion  of  situational  awareness  and  is  a  good  place  to  start  if  you 
are  interested  in  that  larger  discussion. 
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specific  geographic  boundaries,  but  its  significance  comes  back  to  the  comprehension  of 
relevant  information  that  could  potentially  improve  a  decision  that  needs  to  be  made  at  a 
specific  time  and  place.  Thus,  we  can  say  that  infonnation  superiority  facilitates 
situational  awareness,  which,  in  turn,  facilitates  decision  making.  Figure  1  is  a  graphic 
description  of  the  DMSC. 


The  Dynamic  Model  of  Situated  Cognition 


©  Miller  and  Shattuck,  2003 


Figure  1.  Dynamic  Model  of  Situated  Cognition18 

By  way  of  a  brief  description,  Oval  1  contains  all  the  information  in  the 
environment — the  ground  truth,  if  you  will.  Oval  2  is  information  collected  from  the  ground 
truth.  But  Oval  2  is  more  than  simply  a  slice  of  Oval  1;  Oval  2  also  contains  distortions  and 
misrepresentations  of  Oval  1.  Finally,  Oval  3’s  relationship  to  Oval  2  is  similar  to  the  one 
that  Oval  2  has  to  Oval  1,  in  that  additional  errors  can  be  promulgated  as  information  is 
displayed  on  available  command  and  control  systems.  These  errors  can  be  the  result  of 
enemy  actions  as  well  as  friendly  mistakes  and  miscommunications.  At  this  point,  the 


18  Nita  L.  Miller  and  Lawrence  G.  Shattuck,  “A  Process  Model  of  Situated  Cognition  in  Military 
Command  and  Control”,  Proceedings  of  the  2004  Command  and  Control  Research  and  Technology 
Symposium,  San  Diego,  CA.  p.  3. 
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decision  maker  has  not  even  come  into  contact  with  the  information.  Oval  4  contains  the 
data  perceived  by  the  decision  maker,  which  has  been  filtered,  for  better  or  worse,  by  the 
perception  lens  (Lens  A).  Oval  4  is  akin  to  the  Observe  step  in  the  OODA  Loop.  The 
important  nuance  here  is  that  errors  may  already  exist  in  what  the  decision  maker  is 
observing  since,  in  this  context,  the  observations  are  indirect.  Next,  the  information  flows 
from  Oval  4  to  Oval  5  through  the  decision  maker’s  interpretation  lens;  also  akin  to  the 
Orient  step  of  the  OODA  Loop.  Oval  6  could  be  summarized  as  the  resultant  situational 
awareness  of  the  decision  maker  once  the  information  processing  gets  through  the 
extrapolation  lens  (Lens  C). 

This  DMSC  model  does  an  excellent  job  of  explicitly  and  implicitly  displaying 
the  numerous  factors  that  can  improve  or  degrade  a  decision  maker’s  situational 
awareness,  whether  they  are  related  to  command  and  control,  Intelligence,  Surveillance 
and  Reconnaissance  (ISR)  limitations,  redundant  reporting,  enemy  deception,  human 
error,  or  many  more  factors.  In  doing  this,  it  also  highlights  the  complexity  associated 
with  any  attempt  to  mathematically  model  and  explore  the  contribution  of  information 
superiority.  It  is  also  important  to  note  that  achieving  infonnation  superiority  is  not  an 
objective  goal,  but  one  relative  to  the  state  of  your  adversary.  Vice  Admiral  H.  Denby 
Starling  II,  United  States  Navy,  commander  of  Naval  Network  Warfare  Command, 
described  it  succinctly  this  way: 

ISR  (collecting  infonnation  about  enemy  and  terrain) 

+ 

C4  (keeping  track  of  friendly  info) 

+ 

Infonnation  Operations  (Attacking  enemy’s  information  while  protecting 

friendly  info) 

INFORMATION  SUPERIORITY19 


19  Secretary  of  the  Navy  Guest  Lecture,  Naval  Postgraduate  School,  Monterey,  CA,  1  April  2008, 
Admiral  Denby  Stargill,  United  States  Navy  (personally  attended  by  the  author).  This  reference  is  a 
depiction  of  a  concept  presented  on  one  of  Adm.  Sturgill's  slides  but  is  not  meant  as  a  word  for  quotation. 
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Thus,  this  formulation  does  not  imply  that  the  fog  of  war  can  be  eliminated,  but 
that  a  gap  can  be  achieved  between  our  adversary’s  ability  to  use  infonnation  and  our 
own.  The  result  should  be  increased  situational  awareness  for  us  and  a  decreased 
situational  awareness  for  them,  which,  in  turn,  should  result  in  better  decisions  and  a 
better  outcome  from  our  perspective. 

C.  MODELING  APPROACH  AND  SCOPE 

The  single  most  important  factor  in  model  development  to  explore  these  concepts 
was  having  a  model  that  involved  a  decision  maker  in  an  adversarial  situation.  Following 
this  requirement,  there  was  also  a  preference  for  a  closed  loop  model  in  order  to  take 
advantage  of  modem  computing  power  for  conducting  numerous  replications  of  a 
stochastic  process.  Additionally,  simplicity  was  an  important  factor  in  model 
development,  with  the  idea  of  examining  relatively  simple  decision-making  situations  at  a 
high  level  of  abstraction  as  a  starting  point  for  exploration.  This  led  to  the  development 
of  a  model  largely  based  on  game  theory  and  the  concepts  behind  two-person  zero  sum 
(TPZS)  games.20  The  idea  behind  a  TPZS  game  is  that  any  benefit  to  one  player  results 
in  the  direct  detriment  of  the  other  player.  The  applicability  to  conventional  warfare  is 
immediately  apparent,  where  often  two  sides  are  competing  over  various  objectives  and 
just  about  anything  beneficial  to  one  side  is  detrimental  to  the  other. 

Game  Theory,  and  specifically  the  TPZS  game,  has  been  used  before  to  address 
questions  related  to  information  superiority.  Dr.  Jerome  Bracken  and 
Dr.  Richard  Darilek  of  the  RAND  Arroyo  Center  used  game  theory  to  examine  the  value 
of  information  and  its  contribution  to  outcomes.21  Their  work  was  expanded  by 
Lieutenant  Commander  Gary  McIntosh,  United  States  Navy,  as  a  student  at  the 
Naval  Postgraduate  School.22  In  both  of  these  research  efforts,  the  research  was  limited 
strictly  to  the  TPZS  game.  Additionally,  Bracken  and  Darilek  did  not  incorporate  the 


20  Philip  D.  Straffin,  Game  Theory’  and  Strategy’,  The  Mathematical  Association  of  America,  1993. 

21  Jerome  Bracken  and  Richard  E.  Darilek,  “Information  Superiority  and  Game  Theory:  The  Value  of 
Information  in  Four  Games,”  PHALANX,  Vol.  31,  No.  4,  1998,  pp.  6-7,  33-34. 

22  Gary  A.  McCintosh,  “Information  Superiority  and  Game  Theory:  The  Value  of  Varying  Levels  of 
Information,”  Master’s  Thesis,  Naval  Postgraduate  School,  Monterey,  CA,  March  2002. 
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aspect  of  information  quality  in  their  study.  McIntosh  partly  improved  on  their  study  in 
this  area,  but  only  considered  varying  information  accuracy.  While  this  present  research 
seeks  to  address  questions  similar  to  those  posed  in  previous  research,  the  model  used 
here — while  still  drawing  on  TPZS  game  principles  that  will  be  described  in  more  detail 
in  Chapter  II — is  notably  different. 

Applying  a  game  theory  construct  to  these  questions  represents  a  significant  level 
of  abstraction  of  the  situational  awareness  and  decision  process  described  previously. 
Error  in  situational  awareness  will  be  introduced  by  degrading  the  information  quality, 
while  the  introduction  of  error  from  other  sources  will  be  ignored.  Additionally,  not  all 
the  contributors  to  infonnation  superiority  (ISR;  Command,  Control,  Communications, 
and  Computers  (C4);  10)  will  be  considered  as  such,  but  a  general  concept  of  information 
superiority  will  be  applied  where  the  decision  maker  primarily  gains  an  information 
advantage  through  the  collection  of  information,  while  the  ability  to  process  and  protect 
this  information  is  assumed.  The  aim  is  to  get  at  the  essence  of  the  contribution  of 
information  superiority  and  some  of  the  challenges  with  achieving  information 
superiority,  while  assuming  that  the  decision  maker  has  the  wherewithal  to  turn  this 
information  superiority  into  decision  superiority  and  an  actual  operational  advantage. 

With  the  background  set  in  place,  Chapter  II  discusses  the  details  of  the  selected 
modeling  approach.  Additionally,  the  mathematical  principles  involved  are  laid  out  with 
a  detailed  discussion  of  the  model  logic.  This  is  followed  in  Chapter  III  by  a  brief 
explanation  of  experimental  design,  how  and  why  it  was  used  in  this  research,  and  the 
specifics  of  the  design  used  for  this  investigation.  Chapter  IV  covers  the  analysis  of  the 
data  collected  in  the  experiment,  with  conclusions  and  recommendations  for  future 
studies  covered  in  Chapter  V. 
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II.  MODEL  DESCRIPTION 


A.  HOW  GAME  THEORY  APPLIES  TO  THE  MODEL 

The  model  was  developed  as  an  extension  of  a  TPZS  game.  The  scenario  is  as 
follows:  there  are  three  objectives  and  two  sides,  red  and  blue.  The  three  objectives  are 
representative  of  the  locations  where  red  and  blue  will  encounter  one  another  in  a  type  of 
engagement.  Blue  is  defending  the  objectives  with  its  available  resources  and  red  is 
trying  to  maximize  the  sum  of  the  difference  between  red  and  blue  at  each  site.  Figure  2 
provides  a  visualization  of  this  concept  (those  familiar  with  TPZS  games  may  recognize 
this  as  a  form  similar  to  a  Blotto  game.)23 

Model  Starting  Point 
Two-Person  Zero  Sum  Game 


=  Total  Red  Score 

Figure  2.  Two-Person  Zero  Sum  Game  Model  Starting  Point 

Red,  the  attacker,  will  deploy  its  units  to  the  objectives  and  blue  will  do  the  same. 
Following  this  action,  we  can  calculate  how  many  more  reds  are  at  each  location. 


23  To  read  more  about  Blotto  games  see  Washburn,  A.R.,  Two-Person  Zero-Sum  Games ,  Institute  for 
Operations  Research  and  the  Management  Sciences,  December  1994. 
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Summing  these  values,  we  get  the  total  red  score.  Of  course,  red  is  trying  to  maximize 
this  score,  while  blue  is  trying  to  minimize  this  score.  Both  red  and  blue  will  have  a 
number  of  strategies  available  to  them  based  on  their  respective  number  of  units.  In  fact, 
the  number  of  strategies  is  equal  to: 

(#  of  units  +  1)  x  (#  of  units  +  2) 

2 

Thus,  if  blue  had  five  units  he  would  have  (6  x  7)  /  2  =  2 1  possible  strategies  to  choose 
from.  So  how  do  blue  and  red  decide  which  strategy  to  select?  To  answer  this  question, 
we  need  to  understand  more  about  solving  TPZS  games. 

A  key  product  to  understanding  potential  outcomes  of  selecting  certain  strategies 
is  what  is  known  as  the  payoff  matrix.  The  payoff  matrix  displays  all  the  outcomes 
associated  with  the  possible  strategies  for  red  and  blue.  Table  1  is  a  payoff  matrix  for  the 
situation  where  red  has  three  units  and  blue  has  two  units,  and  it  shows  all  the  potential 
payoffs  for  red. 


RED/BLUE  PAYOFF  MATRIX  EXAMPLE 


COLUMN 

A 

B 

C 

D 

E 

F 

Blue  Strategies 

ROW 

0  0  2 

0  1  1 

0  2  0 

1  0  1 

1  1  0 

2  0  0 

A 

0 

0 

3 

1 

2 

3 

2 

3 

3 

B 

0 

1 

2 

1 

1 

2 

3 

2 

3 

C 

C/3 

0 

2 

1 

2 

1 

1 

2 

2 

3 

D 

'5b 

0 

3 

0 

3 

2 

1 

3 

2 

3 

E 

<D 

c5 

1 

0 

2 

1 

2 

3 

1 

2 

2 

F 

GO 

1 

1 

1 

2 

1 

2 

1 

1 

2 

G 

<D 

1 

2 

0 

3 

2 

1 

2 

1 

2 

H 

Pi 

2 

0 

1 

2 

2 

3 

1 

2 

1 

I 

2 

1 

0 

3 

2 

2 

2 

1 

1 

J 

3 

0 

0 

3 

3 

3 

2 

2 

1 

Table  1.  Example  Payoff  Matrix 


First,  observe  that  red  has  (4  X  5)  /  2  =  10  available  strategies,  while  blue  has 
(3  X  4)  /  2  =  6  available  strategies.  Now  let  us  assume  that  red  selects  the  strategy  in  row 
C.  This  means  that  red  is  sending  zero  units  to  objective  1,  two  units  to  objective  2  and 
one  unit  to  objective  3.  Blue,  without  knowledge  as  to  what  strategy  red  selects,  goes 
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with  his  own  strategy  of  column  D,  sending  one  unit  to  objective  1,  zero  units  to 
objective  2,  and  one  unit  to  objective  3.  If  we  then  examine  the  intersection  of  row  C  and 
column  D,  we  see  a  value  of  2,  which  is  equivalent  to  the  red  score  calculated  as 
discussed  previously.  Hence,  the  payoff  matrix  displays  all  possible  outcomes  based  on 
the  combination  of  strategies  of  red  and  blue. 

In  many  cases,  as  in  the  present  situation,  there  is  no  single  strategy  that  is  clearly 
superior  to  all  the  rest.  For  example,  the  best  possible  outcome  for  red  is  a  score  of  three,  and 
if  red  had  a  strategy  in  which  he  could  score  a  three  every  single  time,  that  would  be  best  for 
red.  However,  more  often  than  not,  respective  sides  must  select  a  mixture  of  strategies  and 
play  them  with  what  is  called  a  mixed  strategy  based  on  the  concept  of  an  expected  value, 
where  the  expected  value  of  getting  a  certain  payoff  ai,  a2,  ...aj  with  respective  probabilities 

pi,  p2,  ....pk  is  piai  +  p2a2  +  ...  pkaj.24  Within  game  theory,  this  reasoning  is  known  as  the 
expected  value  principle,  which  is  “If  you  know  that  your  opponent  is  playing  a  given  mixed 
strategy,  and  will  continue  to  play  it  regardless  of  what  you  do,  you  should  play  your  strategy 
which  has  the  largest  expected  value. ”2^  The  reason  for  this  is  that  there  exists  a  value  such 
that  red  can  guarantee  it  gets,  on  average,  at  least  this  value  and  blue  can  guarantee  that,  on 

average,  red  gets  no  more  than  this  valued  This  value  is  known  as  the  value  of  the  game. 

To  solve  this  TPZS  game  for  the  value  of  the  game  and  determine  the  optimal  mixed 
strategy  for  red  and  blue,  we  apply  linear  programming  techniques.  To  solve  for  red,  we  first 
create  a  matrix  Ar  by  adding  a  row  to  the  bottom  of  the  payoff  matrix  with  values  all  equal  to 
-1  and  then  transposing  this  matrix  to  get  Ar.  Next,  we  create  a  vector  x  that  consists  of 
variables  Xi,  x2,  ...,  xn  where  n  equals  the  number  of  columns  in  matrix  Ar.  The  value  of  the 
game  is  xn  and  the  mixed  red  strategies  are  the  values  xK  x2,  . .  ,,xn_i.  Thus,  the  linear  program 

is  formed  in  the  following  manner22: 


24  Philip  D.  Straffin,  Game  Theory  and  Strategy,  The  Mathematical  Association  of  America,  1993,  13. 

25  Ibid.,  13. 

26  Ibid.,  13. 

22  This  particular  method  was  found  at  a  Website  operated  by  Elmer  Gerald  Wiens,  Ph.D.  at 
www.egwald.com/operationsresearch/gametheory.php3,  accessed  January  2008. 
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Maximize:  z  =  xn 

Subject  to: 

Arx  >  0; 

n— 1 

=  °; 

1=1 

where  all  x;  are  non-negative. 

If  it  is  unclear  why  this  method  works,  consider  for  a  moment  what  Arx 

represents.  It  is  a  series  of  equations  testing  whether  the  selected  optimal  mixed  strategy 

is  greater  than  or  equal  to  the  expected  payoff  (xn),  which  we  are  trying  to  maximize. 

The  optimal  mixed  strategy  must,  of  course,  meet  all  of  these  constraints  as  well  as  the 

sum  of  the  mixed  strategies  equaling  zero  (constraint  number  two),  since  each  represents 

the  proportion  of  the  number  of  times  the  specific  strategy  would  be  selected.  Using  the 

example  payoff  matrix  from  the  previous  page,  the  problem  written  in  long  form  would 

look  like  this: 

Maximize:  z  =  xn 
Subject  to: 

lxi  +  1x2  +  2x3  +  3x4  +  lx5  +  2x6  +  3x7  +  2xg  +  3xg  +  3xio  —  lxn  >  0; 

2xi  +  1x2  +  1x3  +  2x4  +  2x5  +  1x6  +  2x7  +  2x8  +  2x9  +  3xio  —  lxn  >  0; 

3xj  +  2x2  +  1x3  +  1x4  +  3x5  +  2x6  +  lx7  +  3xg  +  2xg  +  3xio  —  lxn  >  0; 

2xj  +  3x2  +  2x3  +  3x4  +  1x5  +  1x6  +  2x7  +  1x8  +  2x9  +  2xio  —  lxn  >  0; 

3xi  +  2x2  +  2x3  +  2x4  +  2x5  +  1x6  +  lx7  +  2x8  +  1x9  +  2xio  —  lxn  >  0; 

3xj  +  3x2  +  3x3  +  3x4  +  2x5  +  2x6  +  2x7  +  lxs+  1x9  +  lxio  —  lxn  >  0; 

X[  +  X2  +  X3  +  X4  +  X5  +  X6  +  X7  +  X8  +  X9  +  X10  =  0; 

Xi,  x2,  X3;  x4j  x5;  x6,  x7j  x8j  x9j  xio  >  0; 

To  solve  this  program  for  the  blue  mixed  strategies  we  add  a  column  of-l’s  to 
the  right-hand  side  of  the  payoff  matrix  and  then  transpose  this  matrix  to  create  a  new 
matrix  called  Ab.  We  will  use  the  vector  y  to  represent  the  combination  of  blue’s  mixed 
strategy  and  the  value  of  the  game.  This  vector  is  equal  in  length  to  the  number  of  rows 
in  matrix  Ab-  Now,  instead  of  maximizing,  we  minimize  and  change  the  sign  on  all  of  the 
constraints  and  we  have  the  following  linear  program: 

Minimize:  z  =  yn 

Subject  to: 

Aby  <  0; 

n—i 

Za  =  0; 

i=l 

where  all  x;  are  non-negative. 
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Solving  this  problem  will  show  that  yn  is  equal  to  xn,  which  will  always  be  the 
case.  More  importantly,  the  y  vector  will  reveal  the  optimal  mixed  strategy  for  blue. 

Using  these  methods,  a  payoff  matrix,  game  value,  and  optimal  mixed  strategy 
can  be  computed  for  any  combination  of  blue  and  red  values.  As  it  turns  out,  a 
generalization  can  be  made  for  how  both  blue  and  red  should  behave  under  the  conditions 
described  for  this  scenario.  Blue’s  strategy  is  equivalent  to  trying  to  defend  each 
objective  equally.  In  terms  of  the  output  from  the  linear  program,  this  would  mean  that  if 
blue  had  seven  units,  one -third  of  the  time  he  would  play  the  3,2,2  strategy,  one -third  of 
the  time  the  2,3,2  strategy,  and  one-third  of  the  time  the  time  the  2,2,3  strategy.  On  the 
other  hand,  it  is  best  for  red  to  always  to  keep  the  entirety  of  its  units  focused  on  one 
objective.  What  this  means  for  red,  in  terms  of  the  mixed  strategy,  is  that  red  plays  0,0,5 
one-third  of  the  time,  0,5,0  one-third  of  the  time,  and  5,0,0  one-third  of  the  time  given  it 
has  five  units.  These  will  be  the  general  strategies  employed  by  both  red  and 
blue,  respectively. 

B.  EXTENDING  THE  TWO-PERSON  ZERO  GAME  MODEL  INTO  OUR 

EXPERIMENTAL  MODEL 

The  TPZS  game  model  discussed  in  the  previous  sections  assumes  that  both  sides 
take  the  optimal  action,  based  on  the  fact  that  neither  red  nor  blue  can  anticipate  the 
other’s  strategy.  However,  since  we  are  interested  in  looking  at  situations  where  one  side 
obtains  an  information  advantage,  we  need  to  create  a  mechanism  that  provides 
opportunities  for  one  side  to  gain  information  as  to  its  adversary’s  intentions.  Therefore, 
observe  in  Figure  3,  an  illustration  we  will  now  refer  to  as  the  “game  board.” 


Columns 

1  2  3  4  5 


Figure  3.  Graphic  Depiction  of  the  Game  Board 
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On  the  game  board,  both  red  and  blue  begin  with  the  entirety  of  their  force  at  their 
respective  start  locations.  There  will  be  a  detailed  discussion  as  to  the  rules  of  the  game, 
but  for  now  understand  that  red  will  advance  one  column  per  move  until  it  reaches 
column  4.  Blue  will  deploy  its  forces  over  time  into  column  five.  The  final  step  of  the 
game  is  to  calculate  the  number  of  red  that  outnumber  the  blue  in  each  row  between 
columns  four  and  five.  The  purpose  of  having  all  the  red  movements  is  to  provide  an 
opportunity  for  blue  to  collect  information  as  to  red  intentions  as  red  moves  using  a 
sensor  placement  strategy.  Within  this  current  game,  if  blue  has  no  ability  to  collect  any 
information  on  red  as  red  moved  to  column  four,  then  we  have  simplified  the  scenario 
back  to  the  TPZS  game  discussed  in  Section  A.  Hence,  the  point  is  to  see  how  blue 
might  improve  on  the  outcomes  of  the  basic  game  if  provided  a  potential  information 
advantage  under  varying  conditions. 

C.  RULES  OF  THE  GAME 

This  section  will  be  a  detailed  discussion  on  how  the  game  is  played. 

1.  Starting  Conditions 

It  is  important  to  understand  where  both  blue  and  red  start  in  terms  of  their 
understanding  of  the  opposition.  Remember,  we  began  with  the  standard  TPZS  game 
where  each  side  has  no  knowledge  of  what  the  other  side  will  attempt  to  do.  Despite  the 
lack  of  infonnation  about  their  adversary’s  specific  course  of  action,  but  having 
information  as  to  the  range  of  action,  each  side  is  able  to  take  some  rational  action.  In 
red’s  case,  this  is  to  maximize  the  lowest  possible  expected  value  of  the  outcome;  in 
blue’s  case,  this  is  to  minimize  the  highest  possible  expected  value  of  the  outcome.  Now 
we  have  modified  the  situation.  Red  will  not  be  given  any  additional  infonnation,  but 
still  understands  the  basic  rules  of  the  game.  So,  even  though  we  can  assume  he  has 
knowledge  as  to  blue’s  ability  to  collect  information  on  his  movement,  we  will  not 
assume  he  knows  to  what  degree  or  specificity.  Red  will  stick  to  the  three  courses  of 
action  detennined  by  solving  the  linear  program.  From  this  we  conclude  that  red  has  no 
decisions  to  make  other  than  the  detennination  of  the  optimal  mixed  strategy  to  employ 
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over  the  course  of  a  particular  game.  We  can  also  state  that  red  begins  with  knowledge 
about  the  number  of  blues  and  the  restrictions  of  movement  that  pertain  to  blue,  but 
again,  these  will  not  effect  the  manner  in  which  red  plays  the  game. 

For  blue,  the  situation  is  quite  a  bit  different.  First,  blue  will  also  understand  the 
restrictions  placed  on  red  movement.  Additionally,  blue  will  not  always  have  accurate 
knowledge  as  to  the  total  number  of  red  units.  This  will  vary,  based  on  the  accuracy 
variation  parameter.  Blue  will  have  no  knowledge  as  to  what  strategy  red  will  employ, 
but  will  be  restricted  in  trying  to  determine  this,  based  on  the  quality  of  information 
provided  by  his  sensors  and  the  number  of  sensors  themselves.  The  bottom  line  is  that 
blue  will  have  the  opportunity  to  make  adjustments  to  his  strategy.  For  this  reason,  blue 
will  want  to  remain  as  flexible  as  possible  within  the  rules  of  the  game. 

2.  Red  Movement  Restrictions 

During  its  first  turn,  red  is  required  to  move  the  total  of  its  units  from  the  start  point 
on  the  game  board  to  any  positions  desired  in  column  one.  During  subsequent  turns,  it  must 
adhere  to  three  important  rules  governing  its  movement:  1)  during  each  turn  all  red  units 
must  move  out  of  one  column  and  into  another,  higher-numbered  column;  2)  across  dotted 
paths,  red  can  move  a  maximum  of  one  unit;  and  3)  across  solid  paths,  red  can  move  up  to 
the  maximum  number  of  reds  present  in  the  previous  location  in  each  of  the  respective  rows. 
This  means  that  after  red’s  initial  deployment,  all  of  its  units  will  be  in  column  four  in  three 
turns.  This  is  analogous  to  red  re-arraying  its  forces  on  the  battlefield — a  difficult  task  that 
provides  a  purpose  behind  these  movement  restrictions. 

3.  Blue  Movement  Restrictions 

For  blue’s  initial  deployment  from  blue  start,  blue  can  move  as  many  units  into 
column  five  as  desired.  We  are  not  interest  in  testing  whether  blue  can  establish  its 
defense  in  time  for  the  arrival  of  red  units,  which  is  the  reason  blue  is  allowed  to  place  as 
many  units  as  desired  in  its  initial  deployment.  Rather,  the  intent  is  to  examine  how 
much  can  be  gained  by  allowing  blue  to  adjust  its  defense  with  the  advantage  of 
information  superiority.  However,  for  each  subsequent  turn,  blue  can  move  only  a 
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maximum  of  three  units  out  of  the  blue  start  location.  This  represents  a  limitation  on  the 
amount  of  flexibility  that  blue  enjoys.  Additionally,  no  blue  units  can  return  to  the  start 
location  once  deployed.  During  each  turn,  blue  can  move  a  maximum  of  one  unit 
between  each  objective  in  column  five.  What  this  means  for  blue,  in  practice,  is  that  blue 
will  hold  back  the  maximum  number  of  units  possible  in  order  to  maintain  the  maximum 
level  of  flexibility,  in  the  hopes  that  his  sensors  will  provide  some  useful  information  as 
to  where  red  units  will  be  attacking.  This  is  exactly  the  methodology  that  was 
implemented  in  blue’s  decision  mechanism  for  deploying  forces  within  the  computer 
model.  Blue  has  a  priori  knowledge  of  the  number  of  red  units  but  this  knowledge  is  not 
perfect. 


4.  Basic  Game  Play  Sequence 

The  game  is  played  in  the  following  sequence  (implementation  of  these  steps  will 
be  further  explained  in  Section  F). 

1)  Blue  uses  a  decision  algorithm  to  place  an  assigned  number  of  sensors  on 
the  game  board;  only  one  sensor  can  be  placed  per  location  in  columns 
one  through  three  on  the  game  board. 

2)  Blue  uses  a  decision  algorithm  to  make  initial  deployment  of  blue  units 
into  column  five. 

3)  Red  randomly  selects  one  of  its  three  courses  of  action  to  execute  not 
knowing  blue’s  disposition;  red  moves  the  total  of  red  units  into  column 
one  in  a  manner  that  will  still  allow  it  to  achieve  its  selected  course  of 
action;  the  assignment  of  reds  to  a  specific  location  is  a  purely  stochastic 
process  that  works  within  the  constraints  of  the  red  movement  restrictions. 

4)  If  blue  has  any  sensors  located  in  column  one  and  the  infonnation  is  not 
delayed,  this  infonnation  will  be  provided  to  blue  and  blue  will  adjust  its 
units  in  column  five  subject  to  its  movement  restrictions. 

5)  This  process  will  repeat  until  the  red  units  are  in  column  four;  blue  will 
not  have  any  sensors  in  column  four;  at  this  point,  a  score  is  calculated  for 
red  in  accordance  with  the  calculation  procedure  previously  discussed,  at 
which  point  the  game  has  ended. 
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D.  MODEL  PARAMETERS 


The  model  has  five  parameters:  number  of  red,  number  of  blue,  number  of  blue 
sensors,  accuracy  variation,  and  probability  of  time  delay.  The  number  of  red  and  blue 
should  be  self-explanatory  (however,  it  should  be  noted  that  they  are  integer  values  only). 
Red’s  will  range  from  2  to  25  and  blue  will  range  from  1  to  (3  *  red  -  1).  Once  blue  is 
more  than  three  times  as  large  as  red,  it  can  defend  each  location  without  any  concern 
that  red  will  score  any  points.  Thus,  it  is  not  important  to  explore  this  region. 

Sections  D.l.  through  D.3.  describe  the  reasoning  and  use  of  the  other  parameters 
in  detail. 

1.  Number  of  Blue  Sensors  (Varied  from  1-9) 

In  this  experiment,  blue  is  being  provided  the  information  advantage  in  order  to 
see  how  he  might  improve  over  the  baseline  case  of  information  parity.  Blue  sensors 
have  an  input  range  from  1  to  9.  Only  one  sensor  can  be  placed  at  each  of  the  nine 
discrete  locations  on  the  game  board,  located  in  columns  one  through  four.  Placing  a 
sensor  at  a  particular  location  provides  the  opportunity  for  blue  to  determine  something 
about  the  number  of  red  units  at  that  location  at  a  certain  moment  in  time.  Blue  is  then 
able  to  use  this  information  to  varying  degrees  to  adjust  his  course  of  action.  However, 
the  quality  of  the  information  may  be  impacted  by  the  two  parameters,  accuracy  variation 
and  probability  of  time  delay,  to  be  discussed  in  the  following  section.  The  sensor  has 
the  capability  to  detect  infonnation  with  perfection.  However,  the  accuracy  variation 
parameter  provides  instances  where  errors  will  be  introduced  to  the  information  collected. 

2.  Accuracy  Variation  (Varied  from  1-6) 

The  quality  of  information  is  just  as  important  as  the  information  itself.  Poor 

quality  information  can  actually  do  more  harm  than  good.  To  incorporate  this  aspect  of 

information  quality,  both  parameters  of  accuracy  variation  and  probability  of  delay  have 

been  introduced.  Accuracy  variation  pertains  to  the  standard  error  of  the  normal 

distribution  (mean  =1)  that  is  being  used  to  apply  a  level  of  error  to  the  number  of  units 

at  a  particular  location  being  reported  by  a  sensor.  For  example,  let  us  say  we  want  the 

error  to  be  added  according  to  a  standard  normal  distribution.  To  implement  this,  we 
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would  input  a  one  for  our  accuracy  variation  parameter.  As  a  result,  every  time  a  sensor 
made  a  detection,  a  random  number  would  be  generated  from  a  normal  distribution 
(mean  =  0,  variance  =  1).  If  a  -1.237  is  generated,  everything  to  the  right  of  the  decimal 
is  eliminated  (this  is  not  rounding)  and  a  -1  would  be  added  to  whichever  value  the 
sensor  detected.  Thus,  to  increase  the  likelihood  of  greater  error  being  reported  by 
sensors,  we  increase  the  accuracy  variation  parameter.  Table  2  provides  a  sample  of  how 


the  accuracy  might  vary,  based  on  the  accuracy  variation  parameter. 


Accuracy 

Variation 

Probability 
of  Accurate 
Information 

Probability 
Information 
is  off  by  +/- 

1 

Probability 
Information 
is  off  by  +/- 
2 

Probability 
Information 
is  off  by  +/- 
3 

Probability 
Information 
is  off  by  +/- 
4 

Probability 
Information 
is  off  by 
more  than 
+1-4 

1 

27% 

<1% 

<1% 

<1% 

2 

18% 

9% 

3% 

2% 

3 

26% 

19% 

13% 

9% 

4 

20% 

16% 

14% 

11% 

5 

16% 

15% 

14% 

12% 

11% 

32% 

6 

13% 

13% 

12% 

12% 

10% 

40% 

Note:  These  percentages  have  been  rounded  to  the  nearest  percent. 

Table  2.  Probability  Distribution  Based  on  the  Accuracy  Variation  Parameter 


3.  Probability  of  Time  Delay  (0.0  -  0.8) 

As  you  have  probably  surmised,  this  parameter — which  has  a  range  from 
0  to  0.8 — affects  the  timeliness  of  the  sensor  report.  A  value  of  0.5  would  mean  that 
there  is  a  50%  chance  that  a  delay  will  be  incurred.  The  length  of  the  delay  is  always 
only  one  turn.  Thus,  a  report  that  blue  was  expected  during  his  current  turn  will  not  be 
available  until  the  following  turn.  Both  the  probability  of  time  delay  and  the  accuracy 
variation  apply  uniformly  to  each  sensor  being  employed  within  a  specific  game  (i.e., 
each  sensor  will  have  the  same  probability  of  delay  and  the  same  accuracy  variation). 
This  was  done  primarily  for  the  sake  of  simplicity. 

E.  MODEL  OUTPUTS 

For  each  specific  set  of  inputs,  the  game  will  run  for  a  predetermined  number  of 


replications,  which  will  constitute  one  run  of  the  model.  For  each  run,  the  model  will 
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output  a  low  score,  high  score,  and  the  average  score  for  red.  Figure  4  is  a  visual 
summary  of  the  inputs  and  outputs  of  the  model. 


Inputs 

#  of  Red  (2  -  25) 

#  of  Blue  (1  -  74) 

#  of  Sensors  (1  -  9) 
Probability  of  Time  Delay 

(0-0.8) 

Accuracy  Probability 
(0-6) 


MODEL 


Outputs 
Red  Score 


Figure  4.  Model  Inputs  and  Outputs 

F.  MODEL  IMPLEMENTATION 

This  model  has  been  implemented  in  Java  with  the  addition  of  LPSolve,  which  is 
a  freeware  linear  program/mixed-integer  program  (LP/MIP)  solver  that  is  compatible 
with  Java.  The  following  section  discusses  specific  parts  of  the  implementation  of  the 
game  and  provides  more  details  of  the  specific  functionality  of  the  general  game 
descriptions  provided  in  Sections  C,  D,  and  E. 

1.  Red’s  Movement 

Red’s  movement  is  governed  by  a  stochastic  process  and  since  red  is  not  making 
any  decisions  based  on  the  progression  of  the  game,  all  of  red’s  movements  are 
determined  at  the  beginning  of  the  game.  Hence,  if  red  begins  the  game  with  five  units 
and  selects  the  0,5,0  course  of  action,  we  know  what  the  disposition  of  red  in  column  four 
will  be  and  can  build  the  game  board  backwards  from  there.  Given  that  there  are  a 
positive  number  of  red  units  at  a  specific  location,  there  is  a  fixed  probability  that  red  will 
send  a  red  unit  across  one  of  the  possible  dotted  lines  on  the  game  board.  The 
probabilities  are  weighted  toward  the  location  of  the  preponderance  of  the  red  units  in 
attempt  to  get  good  dispersion  of  the  red  units  across  the  game  board.  Figure  5  contains 
some  example  game  board  builds  given  a  0,5,0  course  of  action  by  red. 
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Figure  5.  Game  Board  Examples 

The  game  board  is  then  manipulated  to  calculate  the  appropriate  score  for  red 
depending  on  how  blue  ends  up  arraying  itself  in  column  five. 


2.  Blue’s  Sensor  Decision  Algorithm 

It  is  important  that  an  effort  be  made  to  ensure  that  blue  is  using  its  sensors  in  the 
best  possible  manner  so  as  to  maximize  its  infonnational  advantage.  In  doing  so,  we 
ensure  that  poor  use  of  sensors  does  not  confound  our  results.  However,  detennining  an 
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optimal  sensor  array  for  a  given  situation  (when  there  were  more  nodes  than  sensors) 
proved  an  intractable  task  for  this  research,  even  when  the  information  was  essentially 
perfect  (i.e.,  no  delay  and  no  erroneous  information).  Using  simulation,  an  attempt  was 
made  to  determine  if,  for  a  given  number  of  red,  blue,  and  blue  sensors,  an  optimal  sensor 
array  could  be  found.  After  conducting  even  1,000  replications  for  a  given  design,  the 
vast  majority  of  design  points  did  not  converge  on  an  optimal  sensor  array.  Therefore,  it 
seemed  best  to  set  up  a  smaller  simulation  within  the  overarching  model  to  select  an 
appropriate  senor  array  for  blue.  While  the  selection  may  not  be  optimal,  it  should,  at 
least,  be  reasonable.  This  smaller  simulation  works  similarly  to  the  larger  simulation, 
however,  the  difference  lies  in  the  fact  that  for  a  given  number  of  red,  blue,  and  blue 
sensors,  a  specific  game  will  be  played  varying  the  other  two  input  parameters 
(Probability  of  Time  Delay,  Accuracy  Variation)  stochastically  within  the  smaller 
simulation,  while  testing  every  possible  sensor  configuration  available  to  blue  each  for  50 
replications. 

By  way  of  example,  let  us  say  we  are  considering  a  scenario  where  there  are  five 
red,  five  blue,  and  four  sensors.  Blue  must  decide  where  the  four  sensors  will  be  placed. 
There  are  9  (possible  locations  among  which  to  choose)  choose  4  (locations  that  will 
actually  be  selected)  possible  ways  that  blue  can  emplace  four  sensors.  In  this  case,  that 
number  equals  126  possible  sensor  strategies  among  which  to  choose.  Each  of  these  126 
possibilities  will  be  run  through  a  smaller  simulation  of  this  model  50  times,  with  an 
average  score  being  computed  for  each  possible  sensor  strategy.  The  strategy  with  the 
highest  average  will  be  the  one  selected  by  blue  to  be  used  against  red  in  the  larger  game. 
Within  each  of  these  smaller  simulations,  the  values  of  the  Accuracy  Variation  parameter 
and  the  Probability  of  Time  Delay  will  be  varied  stochastically  for  each  and  every 
subgame.  Two  important  assumptions  will  be  made  concerning  the  type  of  distributions 
that  apply  to  each  of  these  varying  parameters.  First,  blue  assumes  the  variation  of  the 
probability  of  delay  will  be  between  0  and  0.8  and  that  accuracy  variation  will  bebetween 
1  and  6  per  the  parameter  ranges  of  the  model.  Second,  there  is  a  presumption  on  the  part 
of  blue  that,  most  of  the  time,  information  will  be  on  time  and,  a  majority  of  the  time,  the 
information  will  be  close  to  accurate. 
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Therefore,  the  distribution  of  the  Probability  of  Time  Delay  will  follow  the 
nonnal  distribution  with  mean  =  0,  and  standard  deviation  =  0.15.  Any  negative  values 
drawn  from  this  normal  are  converted  to  positive  numbers.  This  means  that  65%  of  the 
time,  the  probability  of  delay  will  be  15%  or  less,  95%  of  the  time  it  will  be  30%  or  less, 
and  99%  of  the  time  it  will  be  45%  or  less. 


The  accuracy  variation  parameter  will  vary  between  the  values  of  0  and  6  for  the 
subgame.  The  value  of  the  accuracy  parameter  for  a  specific  sub  game  will  be  randomly 
selected  based  on  the  algorithm  described  in  Figure  6. 

Draw  a  uniform  random  number  between  0  and  1 ; 


Accuracy  Variation  =  < 


0  if  U  <  0.05 

1  if  0.05  <  U  <  0.20 

2  if  0.20  <  U  <  0.55 

3  if  0.55  <  U  <  0.75 

4  if  0.75  <  U  <  0.90 

5  if  0.90  <  U  <  0.95 

6  if  U  >  0.95 


Figure  6.  Probability  Distribution  for  the  Accuracy  Variation  Parameter  Within 

the  Subgame 

This  represents  the  idea  that  75%  of  the  time  blue  expects  to  receive  fairly 
accurate  information  (an  accuracy  variation  parameter  less  than  or  equal  to  3). 


3.  Blue’s  Movement  Decision  Algorithm 

The  procedure  for  detennining  how  blue  will  move  into  column  five  is  probably 
the  most  complex  process  of  the  entire  model,  and  involves  numerous  comparisons  and 
updates.  As  mentioned  before,  blue  desires  to  maintain  maximum  flexibility  due  to  the 
fact  that  blue  has  the  capability  to  gain  infonnation  and  wants  to  be  in  a  position  to  react. 
For  this  reason,  blue  always  elects  to  keep  the  maximum  number  of  blue  units  at  the  start 
location  as  long  as  possible,  provided  none  are  left  there  at  the  end  of  each  game.  For 
this  reason,  blue  will  only  deploy  the  maximum  of  either  the  number  of  blue  minus  9,  or 
0,  during  the  initial  deployment.  The  9  comes  from  the  fact  that  during  each  turn,  blue 
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has  the  opportunity  to  move  3  blue  from  the  start  location.  Since  blue  has  three  turns 
after  the  time  of  the  initial  deployment,  blue  will  have  the  opportunity  to  deploy  the  9 
units  not  yet  committed.  The  idea  is  not  to  commit  forces  before  it  is  necessary.  For 
example,  let  us  say  blue  begins  with  12  units;  blue  will  initially  deploy  max(12-9,0)=3 
units.  Since  blue’s  general  strategy,  lacking  any  other  information,  is  to  defend  the 
objectives  equally,  blue  will  place  one  unit  at  each  objective  in  this  case. 

For  future  deployments  of  blue  units,  blue  will  look  to  update  its  infonnation 
concerning  red’s  intentions.  To  do  this,  blue  keeps  track  of  two  pieces  of  information 
about  red:  1)  the  maximum  number  of  reds  possible  at  each  objective,  and  2)  the 
minimum  number  of  reds  required  at  each  objective.  These  two  values  are  based  on  red’s 
movement  restrictions.  Figure  7  provides  an  analysis  of  how  this  calculation  works, 
based  on  the  number  of  reds  identified  at  a  certain  location. 

Delta  Values  for  Red  Locations 


Columns 

1  2  3  4  5 


Figure  7.  Delta  Values  Based  on  Red  Locations 

The  numbers  in  columns  one  through  four  represent  the  amount  of  change  that 
can  occur  (plus  or  minus)  between  the  number  of  reds  at  a  current  location  and  the 
number  of  reds  at  the  final  location,  respective  to  each  row.  Thus,  let  us  say  that  15  reds 
are  determined  to  be  in  row  2,  column  2.  The  delta  value  for  that  location  is  4;  therefore, 
the  maximum  possible  reds  that  can  be  in  row  2,  column  4  is  19.  Additionally,  there 
must  be  at  least  1 1  since  red  can,  at  maximum,  move  4  units  from  row  2  within  the  next 
two  turns.  Now  let  us  consider  the  case  where  blue  only  has  one  sensor  and  it  is  located 
at  row  2,  column  2  (the  location  currently  under  discussion);  clearly  an  update  for  the 
maximum  can  be  made  for  the  objective  in  row  2,  but  what  of  the  others?  Well,  since 
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blue  has  calculated  the  minimum  possible  at  the  objective  in  row  2  to  be  11,  he  can 
subtract  this  from  the  total  number  of  reds.  If  we  say  the  total  number  of  reds  is  25,  that 
means  that  there  are,  at  most,  25  -  11  =  14  possible  red  units  at  either  of  the  other  two 
objectives. 

At  this  point,  we  can  develop  a  vector  V  of  the  maximum  possible  red  units 
possible  at  each  objective.  Vi  will  represent  the  maximum  red  units  possible  at  the  row  1 
objective  and  so  on.  It  is  this  vector  that  governs  blue  deployments  as  a  weighting 
scheme.  If  Vi  through  V3  are  equal  (as  is  the  case  at  the  time  of  initial  deployment  when 
they  all  equal  the  total  number  of  reds),  then  each  objective  has  equal  weight  and  blue 
attempts  to  defend  each  equally.  If  this  vector  remains  unchanged  throughout  each  turn, 
blue  will  evaluate  the  delta  between  the  maximum  possible  at  a  particular  objective  and 
the  current  number  of  blues  deployed  to  that  objective.  Blue  seeks  to  keep  these  deltas  at 
equal  values.  So,  if  blue  has  more  units  deployed  to  a  certain  location  above  the 
maximum  number  possible,  it  will  begin  to  move  those  units  to  other  locations — the 
priority  being  to  locations  with  the  largest  delta  between  the  maximum  number  of  reds 
possible  and  the  number  of  blues  deployed  to  that  location.  It  is  probably  helpful  to  go 
through  an  example.  Let  us  consider  a  situation  where  red  has  7  units,  blue  has  12  units, 
and  blue  has  2  sensors.  This  situation  is  depicted  in  Figure  8,  where  the  squares  represent 
the  locations  of  the  blue  sensor. 
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Figure  8.  Game  Board  Depicting  Location  of  Blue  Sensors,  Number  of  Red  Units, 

and  Number  of  Blue  Units 

In  this  example,  blue  makes  its  initial  deployment  and  then  red  does  the  same 
with  the  result  seen  in  Figure  9. 


1 


co 

£ 


o 


Oh 


2 


3 


Columns 


1  nO.  T 

©r.O-JZh© 

X  X — X 

OrJ©r,©X© 

X  X  X 

®-F>©  ^© 


Figure  9.  Depiction  of  Game  Board  Following  Initial  Deployments 


If  blue  had  any  sensors  in  column  one,  he  might  be  able  to  make  an  update  to 
vector  V;  as  it  is,  the  vector  remains  (7,  7,  7)  and  blue  makes  his  next  deployment  shown 
in  Figure  10. 
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Figure  10.  Depiction  of  Game  Board  Following  Turn  One 


25 


Now,  assuming  there  was  no  delay  and  the  information  was  accurate,  a  blue 
sensor  picks  up  the  fact  that  there  are  five  red  units  located  in  row  three  and  column  two. 
Due  to  the  movement  restriction  placed  on  red,  there  are  a  maximum  of  seven  red  units 
that  can  show  up  at  the  objective  located  in  row  three.  Therefore,  blue  will  not  change 
the  value  of  V3.  However,  V)  and  V2  can  both  be  decreased  since  the  minimum  value  of 
red  units  possible  at  the  objective  located  in  row  three  is  three.  That  is,  three  red  units 
that  cannot  be  used  at  the  other  two  objective  locations.  Thus,  blue  can  reduce  Vi  and  V2 
by  three.  This  means  V  now  equals  (4,4,7).  As  a  result,  of  the  next  three  units  that  blue 
deploys,  two  will  go  to  row  three,  column  five  and  the  last  one  will  be  randomly 
distributed  among  the  three.  Thus,  as  blue  makes  its  next  deployment  and  red  makes  its 
next  move,  the  game  board  could  look  like  Figure  11. 
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Figure  1 1 .  Depiction  of  Game  Board  Following  Turn  Two 

In  this  next  step,  blue  again  has  a  sensor  in  play  and  detennines  that  there  are  zero 
red  units  located  at  row  one,  column  three.  The  update,  in  this  case,  is  only  made  to  Vi 
since  the  minimum  value  is  of  no  value  (it  equals  zero  in  this  instance)  and  V  becomes 
(1,  4,  7).  Blue  will  now  try  to  move  blue  units  away  from  the  objective  in  row  one 
because  it  determines  it  has  too  many,  and  then  the  final  three  units  will  be  deployed.  It 
can  only  move  one  away  per  turn,  so  blue  will  finish  this  game  with  two  blue  units  at  the 
objective  in  row  one.  Again,  the  first  two  blue  units  deployed  will  go  to  row  three  in 
order  to  even  up  the  gap  between  V  and  the  current  disposition  of  blue  units.  The  final 
blue  unit  will  be  randomly  assigned  between  rows  two  and  three  since  these  are  the  only 
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locations  blue  deems  in  need  of  additional  units.  Thus,  the  game  board  could  turn  out  as 
displayed  in  Figure  14.  In  this  case,  it  turns  out  red  will  get  a  score  of  1,  although  it  was 
possible  that  red  could  have  scored  zero  if  the  final  blue  unit  would  have  gone  to  row 
three. 


Columns 


1  2  3  4  5 


Figure  12.  Depiction  of  Game  Board  Following  Turn  Three 


Of  course,  all  of  these  calculations  are  being  influenced  by  the  Accuracy 
Variation  and  the  Probability  of  Time  Delay  parameters.  If  there  is  a  delay  in  blue 
gaining  information,  blue  loses  the  opportunity  to  take  advantage  of  the  potential  value  of 
that  infonnation.  However,  prior  to  the  next  turn,  vector  V  is  updated  first,  based  on  the 
delayed  information,  and  then  it  is  updated  based  on  the  availability  of  any  new 
information,  and  then  blue  makes  its  next  deployment  of  units.  There  is  no  direct 
resolution  for  any  potential  conflicting  or  ambiguous  information.  By  virtue  of  how  the 
algorithm  is  constructed,  some  deference  is  given  to  the  latest  information,  but  there  is  no 
direct  attempt  to  resolve  any  potential  ambiguity.  For  example,  given  a  delay  from 
column  2  to  column  3  for  a  specific  location,  blue  will  first  adjust  his  disposition  based 
on  the  delayed  information  and  then  use  the  current  information.  However,  let  us  say  that 
the  delayed  information  indicated  the  presence  of  10  red  in  the  previous  location  and  the 
current  infonnation  says  that  there  are  now  2  red  in  the  subsequent  location.  Blue  would 
conclude  that  one  or  both  of  the  sensor  information  provided  has  an  error  based  on  the 
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red  movement  restriction.  Red  could  never  decrease  by  8  units  between  two  locations  in 
the  same  row.  However,  this  model  does  not  attempt  to  resolve  this  observable  error  but 
simply  allows  the  algorithm  to  continue. 

With  the  ground  laid  in  understanding  the  model,  Chapter  III  will  discuss  the 
experimental  design  meant  to  explore  the  model  and  move  the  study  toward  the 
development  of  interesting  insights  concerning  information  superiority. 
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III.  EXPERIMENTAL  DESIGN 


A.  INTRODUCTION 

With  our  model  fully  implemented  using  the  JAVA  computing  language,  we  now 
want  to  take  advantage  of  modern  computing  power  to  run  the  simulation  many  times 
across  the  spectrum  of  input  parameters.  The  purpose  of  experimental  designs  is  to  help 
us  explore  our  model  more  efficiently.28  When  dealing  with  stochastic  models  with  even 
a  small  number  of  input  parameters,  exploring  the  “Landscape  of  Possibilities” — to  steal 
the  theme  from  the  2008  Data  Farming  Workshop — can  be  a  challenging  task.  Where  are 
the  points  of  interest  and  what  areas  require  a  more  detailed  search?  How  is  a  simple 
overview  of  the  “landscape”  accomplished? 

Let  us  review  the  input  parameters  and  ranges  for  each  parameter  of  our  model, 
which  was  discussed  in  detail  in  Chapter  II.  Table  3  lists  the  five  input  parameters  and 
their  ranges. 


Parameter 

Range 

Decimals 

Red  Units 

2-25 

0 

Blue  Units 

1  -74 

0 

Blue  Sensors 

1-9 

0 

Probability  of  Time  Delay 

0.00-0.80 

2 

Accuracy  Variation 

0.0 -6.0 

1 

Table  3.  Table  3.  List  of  Model  Parameters  with  Respective  Ranges 

An  important  point  to  make  about  the  number  of  blue  units  is  that  for  a  given 
number  of  red,  the  blue  units  will  only  range  from  1  to  (3  *  number  of  red  -  1).  Thus, 
there  are  950  possible  combinations  of  red  and  blue  units.  From  Figure  15,  we  see  that 
there  are  nine  possible  variations  for  the  sensors,  8 1  for  the  probability  of  time  delay,  and 
60  for  the  accuracy  variations.  This  means  that  there  are  950  *9*81*  60  =  41,553,000 
different  design  points  or  distinct  inputs  variations  that  we  can  input  to  the  model.  If  we 


28  Susan  M.  Sanchez,  “Work  Smarter,  Not  Harder:  Guidelines  for  Designing  Simulation 
Experiments,”  Proceedings  of  the  2006  Winter  Simulation  Conference,  pp.  47-57. 
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wanted  to  do  only  ten  replications  on  each  design  point,  we  would  exceed  400  million 
runs  of  our  model.  That  is  not  only  a  lot  of  data  to  analyze,  but  it  may  exceed  available 
computing  resources.29  Therefore,  we  apply  a  design  of  experiments. 

A  specific  design  is  a  matrix  of  the  design  points  that  will  be  input  into  the  model 
where  the  columns  of  the  matrix  are  variations  of  specific  parameters  and  each  row 
contains  the  combination  of  parameter  values  for  a  specific  design  point.  Ideally,  the 
design  points  would  be  spread  apart  from  each  other  and  the  columns  of  our  matrix 
would  be  orthogonal,  allowing  us  a  better  opportunity  to  explore  the  multidimensional 
space.  By  way  of  a  simple  example,  let  us  say  we  are  trying  to  map  a  certain  section  of 
the  ocean  floor;  a  three-dimensional  space.  A  single  data  point  will  give  us  an  x,  y,  and  z 
coordinate  for  a  specific  location.  If  we  only  had  the  ability  to  collect  ten  data  points,  we 
would  want  an  optimal  spread  of  those  ten  points  to  get  the  best  overview  of  the  terrain 
possible.  As  we  increase  the  number  of  data  points  collected,  it  should  be  obvious  that 
the  resolution  of  our  map  of  the  ocean  floor  increases.  However,  we  want  to  increase  the 
resolution  uniformly  across  the  entire  space;  thus,  we  want  to  maintain  this  optimal 
spread  of  our  data  points.  This  is  the  concept  behind  the  use  of  nearly  orthogonal  Latin 
hyper-cubes  (NOLH)  to  build  an  experimental  design  that  has  good  space-filling 
properties,  while  keeping  the  correlation  between  the  columns  to  a  minimum.30 
Additionally,  it  helps  us  avoid  having  large  unpredictable  slopes  or  discontinuities.  A 
review  of  the  references  will  provide  a  more  detailed  explanation  of  both  the  conceptual 
underpinnings  of  experimental  design  and  the  mathematics  associated  with  building  an 
NOLH. 

B.  DESIGN  SPECIFICS 

This  experiment  was  built  on  a  crossed-design  method,  implying  a  crossing  of 
two  design  methodologies.  Due  to  the  peculiar  relationship  between  the  number  of  blue 
to  the  number  of  red,  a  complete  enumeration  was  used  for  the  combination  of  these  two 

29  In  the  case  of  this  experiment,  the  design  being  applied  took  less  than  five  hours  which  represented 
1,564,200  runs  of  the  model.  However,  the  simulation  was  run  on  a  cluster  of  32  computers,  which  greatly 
reduced  the  time  required. 

30  Thomas  M.  Cioppa  and  Thomas  W.  Lucas,  “Efficient  Nearly  Orthogonal  and  Space-Filling  Latin 
Hypercubes,”  Technometrics,  Volume  49,  Number  1,  February  2007. 
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parameters.  This  means  that  the  rest  of  the  experiment  was  built  around  the 
948  combinations  of  red  and  blue.  For  each  of  these  combinations,  an  NOLH  design  was 


applied  using  the  remainder  of  the  parameters.  The  NOLH  design  is  shown  in  Table  4. 


low 

1 

1 

0 

1 

0 

0 

1 

0 

0 

high 

9 

6 

0.8 

9 

6 

0.8 

9 

6 

0.8 

decimals 

0 

1 

2 

0 

1 

2 

0 

1 

2 

factor 

Sensors 

AccVar 

ProbDel. 

Sensors 

AccVar 

ProbDel. 

Sensors 

AccVar 

ProbDel. 

9 

1.5 

0.35 

3 

5.3 

0.5 

7 

2.8 

0.8 

8 

6 

0.1 

4 

2.8 

0.15 

7 

1.9 

0.73 

8 

3.2 

0.73 

2 

0.2 

0.48 

7 

0.2 

0.25 

6 

5.4 

0.8 

4 

5.6 

0.13 

8 

0.4 

0.35 

9 

1.2 

0.38 

3 

4.1 

0.58 

4 

3.4 

0.05 

9 

5.7 

0.25 

3 

2.6 

0.18 

2 

5.3 

0.03 

7 

3.3 

0.78 

3 

0 

0.53 

4 

5.4 

0.7 

5 

4.4 

0.75 

4 

5.4 

0.2 

3 

6 

0.43 

6 

2.3 

0.18 

5 

4.3 

0.25 

1 

1.1 

0.48 

7 

4.3 

0.23 

7 

1.3 

0.43 

2 

2.3 

0.68 

7 

2.1 

0.6 

9 

2.1 

0.05 

2 

0.9 

0.3 

7 

4.6 

0.53 

9 

4.5 

0.78 

5 

2.4 

0.18 

6 

1.8 

0.15 

6 

3.6 

0.1 

9 

4.7 

0.28 

8 

4 

0.3 

8 

0.9 

0.45 

9 

4.3 

0.2 

6 

1.9 

8 

2.3 

0 

6 

4.5 

0.58 

8 

4.1 

9 

4.9 

0.73 

6 

3.9 

0.65 

5 

3.5 

0.4 

5 

3 

0.4 

5 

3 

0.4 

1 

5.5 

0.45 

8 

0.8 

0.3 

4 

3.2 

2 

1 

0.7 

6 

3.2 

0.65 

3 

4.1 

0.08 

2 

3.8 

0.08 

8 

5.8 

0.33 

3 

5.8 

0.55 

5 

1.6 

0 

6 

0.4 

0.68 

3 

5.6 

0.45 

2 

5.8 

0.43 

7 

1.9 

0.23 

6 

2.6 

0.75 

1 

1.3 

0.55 

7 

3.4 

0.63 

8 

0.8 

0.78 

4 

3.7 

7 

6 

0.28 

6 

0.6 

0.1 

5 

2.6 

6 

0.6 

0.6 

7 

0 

0.38 

4 

4.8 

5 

1.7 

0.55 

9 

4.9 

0.33 

3 

2.7 

0.58 

4 

4.7 

0.38 

8 

3.8 

0.13 

3 

4.9 

0.2 

1 

3.9 

0.75 

8 

5.1 

0.5 

3 

2.4 

2 

1.5 

0.03 

5 

3.6 

4 

5.2 

5 

2.4 

0.7 

1 

1.3 

2 

3 

0.5 

2 

5.1 

0.35 

2 

1.7 

4 

5.1 

0.13 

2 

3.8 

0.8 

4 

1.5 

3 

2.9 

0.33 

1 

1.1 

0.08 

5 

2.1 

Table  4.  NOLH  for  3  Model  Parameters 
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There  are  99  design  points  that  have  been  placed  in  adjacent  columns  of  33  each 
for  the  convenience  of  showing  the  design  all  on  one  page.  Each  of  the  99  design  points 
for  Number  of  Sensors,  Accuracy  Variation,  and  Probability  of  Time  Delay,  were 
combined  with  each  combination  of  red  and  blue  to  create  948  x  99  =  93,852  design 
points.  Each  of  these  design  points  was  replicated  50  times  for  a  total  of  4,692,600  data 
points  at  the  end  of  the  experiment.  A  full  design  would  have  meant  35,245,692  design 
points  alone. 

Before  moving  on  to  the  analysis  of  the  data,  we  can  examine  a  few  items 
regarding  our  experimental  design.  First,  let  us  observe  the  correlation  of  the 
three-parameter  NOLH  from  Table  4.  Table  5  shows  the  output  from  Excel  ’97  when 
calculating  the  correlation  between  each  column  of  the  respective  factors. 


Column  1 

Column  2  Column  3 

Column  1 

1 

Column  2 

-0.00113 

1 

Column  3 

0.01094 

-0.00279  1 

Table  5.  Column  Correlation  of  NOLH 


Observe  that  the  correlation  between  the  three  columns  is  nonzero,  but  it  is  very 
low,  which  is  what  we  desire.  Second,  Figure  13  shows  a  scatterplot  matrix  of  the  input 
parameters  for  the  31,350  design  points.  We  want  to  focus  on  the  last  three  blocks  that 
display  how  the  33  points  of  the  NOLH  fill  out  the  design  points.  What  we  observe  is 
that  the  33  points  have  a  good  spread  across  the  design  space.  Also  note  the  gapped 
areas,  which  is  not  necessarily  a  point  of  concern.  This  gapped  space  could  be  filled  in 
with  additional  design  points  by  rotating  the  design  if  desired,  but  is  not  essential — 
depending  on  the  conclusion  trying  to  be  drawn.  Moving  forward  with  our  experimental 
design,  Chapter  IV  will  discuss  the  analysis  of  the  data  farmed  in  this  experiment. 
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IV.  DATA  ANALYSIS 


A.  SETTING  THE  COMPARATIVE  BASELINE 

Recall  from  Chapter  II  that  central  to  our  model  are  the  concepts  of  a  two-person 
zero  sum  (TPZS)  game.  Also  discussed  in  Chapter  II  is  the  fact  that  TPZS  games  can  be 
solved  mathematically  to  determine  a  game’s  value,  which  is  equivalent  to  the  expected 
outcome  of  the  game  if  each  player  chooses  their  respective  optimal  strategy.  This 
average,  or  expected,  outcome  will  serve  as  the  basis  of  comparison  as  we  prepare  to 
examine  the  effects  on  the  game  when  blue  is  allowed  an  increasing  number  of  sensors. 
Keep  in  mind,  the  effects  of  information  quality  represented  by  the  probability  of  delay 
and  accuracy  variation  parameters  will  also  be  taken  into  account. 

In  the  context  of  our  model,  the  TPZS  game  serves  as  the  example  of  the  situation 
where  information  parity  exists  between  to  the  two  players  (red  and  blue).  Figure  14  is  a 
compilation  of  all  the  game  values  given  a  specific  number  of  red  and  a  specific  number 
of  blue.  The  number  of  red  units  is  explicitly  labeled  on  the  x-axis,  while  the  set  of  data 
points  over  a  specific  red  unit  label  refer  to  the  game  values  associated  with  the 
respective  number  of  red  units  and  an  increasing  number  of  blue  units  within  each 
grouping.  This  graphical  technique  will  be  applied  regularly  in  the  explanation  of  the 
results.  Figure  14  explains  how  to  infer  the  range  of  blue  units  associated  with  a  given 
number  of  red. 

The  analysis  of  the  data  will  include  the  use  of  linear  regression  models  and 
graphical  representations  of  the  data  provide  by  the  JMP  and  Microsoft  Excel  ’97 
software,  with  conclusions  to  follow  in  Chapter  V. 
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B. 


LINEAR  REGRESSION  MODEL 


Linear  regression  models  are  an  excellent  tool  for  assessing  the  impact  of  each 
parameter,  examining  the  amount  of  variation  within  a  simulation,  and  for  gathering 
insights  about  the  interactions  among  the  various  model  parameters.  In  the  case  of  this 
experiment,  we  have  five  model  parameters:  the  number  of  red  units,  the  number  of  blue 
units,  the  number  of  sensors  (for  blue),  accuracy  variation,  and  probability  of  time  delay. 
The  predictor  variable  used  is  a  value  labeled  the  improved  percent  decrease  in  red  units. 
This  value  is  calculated  as  follows: 

Improved  %  Decrease  =  Expected  %  Reduction  to  Red  Force  -  %  Reduction  to  Red  Force 


where 


„  Number  of  Red  -  Game  Value 

Expected  %  Reduction  to  Red  Force  = - 

Number  of  Red 


and 


%  Reduction  to  Red  Force  = 


Number  of  Red  -  Red  Score 
Number  of  Red 


An  equivalent  mathematical  expression  is: 

T  ,  ^  Red  Score  -  Game  Value 

Improved  %  Decrease  = - 

Number  of  Red 

This  measurement  will  be  used  numerous  times  throughout.  Applying  a  percentage, 
rather  than  examining  raw  red  score  values,  seemed  to  more  meaningful  when  looking 
over  a  variety  of  scenarios  where  the  size  of  red  and  blue  were  varied  significantly.  It  is 
also  important  to  note  that  the  regressions  were  not  accomplished  using  the  more  than 
4.5  million  data  points  representing  the  replication  of  each  of  the  93,582  design  points  50 
times,  but  an  average  red  score  was  computed  for  each  design.  Since  we  are  focused  on 
identifying  significant  trends  in  the  data,  this  also  seemed  a  reasonable  maneuver. 
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A  variety  of  interaction  and  polynomial  levels  were  examined  using  the  step-wise 
regression  feature  within  JMP.  Regression  models  that  incorporated  third  order 
polynomials  and  three-way  interaction  yielded  the  best  results.  Figure  15  is  a  summary 
of  the  regression  model  that  was  fit  to  the  data.  With  29  model  parameters  and  an 
adjusted  R-square  almost  in  line  with  the  actual  R-square,  this  is  a  very  good  model  fit. 
The  model  accounts  for  93%  of  the  variation  in  the  Improved  %  Decrease,  and  all  terms 
are  highly  significant  (p-value<.0005). 
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Summary  of  Fit 

RSquare 

0.930949 

RSquare  Adj 

0.930928 

Root  Mean  Square  Error 

0.028289 

Mean  of  Response 

0.210334 

Observations  (or  Sum  Wgts) 

93852 

Parameter  Estimates 

Term 

Intercept 

NumRed 

NumBlue 

BlueToRed 

NumSensors 

ProbDelay 

AccVar 

(NumRed-17.1392)*(NumBlue-25.7089) 

(NumRed-1 7. 1392)*(NumSensors-5. 121 21) 

(NumRed-1 7.1 392)*(ProbDelay-0.40242) 

(NumRed-1 7.1 392)*(AccVar-3. 17172) 
(NumBlue-25.7089)*(BlueToRed-1 .5) 
(NumBlue-25.7089)*(NumSensors-5. 12121) 
(NumBlue-25.7089)*(ProbDelay-0. 40242) 
(NumBlue-25.7089)*(AccVar-3. 1 7 1 72) 

(BlueToRed-1 ,5)*(NumSensors-5.12121 ) 

(BlueToRed-1.5)*(ProbDelay-0.40242) 

(BlueToRed-1.5)*(AccVar-3.17172) 

(NumBlue-25.7089)*(BlueToRed-1.5)*(NumSensors-5.12121) 
(NumBlue-25.7089)*(BlueToRed-1 ,5)*(ProbDelay-0. 40242) 
(NumBlue-25.7089)*(BlueToRed-1 ,5)*(AccVar-3. 1 7 1 72) 
(NumRed-17.1392)*(NumRed-17.1392) 
(NumRed-17.1392)*(NumRed-17.1392)*(  NumRed- 17. 1392) 
(NumBlue-25.7089)*(NumBlue-25.7089) 
(NumBlue-25.7089)*(NumBlue-25.7089)*(NumBlue-25.7089) 
(BlueToRed-1 ,5)*(BlueToRed-1 .5) 

(BlueToRed-1 ,5)*(BlueToRed-1 ,5)*(BlueToRed-1 .5) 
(NumSensors-5. 12121  )*(NumSensors-5. 12121) 
(NumSensors-5.12121)*(NumSensors-5.12121)*(NumSensor 


Estimate 

Std  Error 

t  Ratio 

Prob>|t| 

1.2081456 

0.006534 

184.91 

0.0000* 

-0.042858 

0.000373 

-114.9 

0.0000* 

0.0273653 

0.000244 

112.17 

0.0000* 

-0.503757 

0.0042 

-120.0 

0.0000* 

0.0046076 

0.000106 

43.59 

0.0000* 

-0.095354 

0.000569 

-167.5 

0.0000* 

-0.018676 

8.054e-5 

-231.9 

0.0000* 

-0.001619 

1.571e-5 

-103.1 

0.0000* 

-0.000705 

1 ,432e-5 

-49.26 

0.0000* 

0.004666 

0.000145 

32.19 

<.0001* 

0.0022858 

2.023e-5 

112.97 

0.0000* 

-0.019267 

0.000168 

-114.7 

0.0000* 

0.0005509 

9.257e-6 

59.51 

0.0000* 

-0.004381 

9.367e-5 

-46.77 

0.0000* 

-0.00063 

0.000013 

-48.17 

0.0000* 

-0.012117 

0.000166 

-72.93 

0.0000* 

0.097274 

0.001681 

57.86 

0.0000* 

0.010758 

0.000235 

45.83 

0.0000* 

-0.000345 

3.268e-6 

-105.5 

0.0000* 

0.003009 

0.000033 

90.98 

0.0000* 

0.0003579 

4.61 7e-6 

77.51 

0.0000* 

0.0002403 

9.642e-6 

24.92 

<.0001* 

0.0000604 

4.807e-7 

125.66 

0.0000* 

0.0006509 

6.096e-6 

106.78 

0.0000* 

-1.905e-6 

3.382e-8 

-56.31 

0.0000* 

-0.003943 

0.001107 

-3.56 

0.0004* 

0.0171338 

0.00031 

55.20 

0.0000* 

-0.002228 

1.735e-5 

-128.4 

0.0000* 

;-5.12121)  0.0008042 

8.388e-6 

95.87 

0.0000* 

Figure  15.  Regression  of  %  Decrease  in  Red  Units  vs.  Model  Parameters 

We  observe  that  all  of  the  parameters  are  included  in  the  first  order  effects  and 
they  are  having  the  impact  we  expect  on  the  improved  percent  decrease  (IPD);  increasing 
red  units  decreases  IPD,  probability  of  delay  and  accuracy  variation  decreases  IPD, 
increasing  blue  units  and  number  of  sensors  increases  the  IPD.  This  model  also 
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incorporates  a  blue-to-red  ratio  that  is  computed  by  dividing  the  number  of  blue  units  by 
the  number  of  red  units.  Intuition  simply  led  to  the  inclusion  of  this  parameter  and  it 
turns  out  that  it  is  significant.  Nothing  stood  out  in  terms  of  the  interaction  terms  or 
polynomial  terms,  but  the  next  question  is,  of  course,  how  substantial  an  impact  do  these 
terms  have  on  the  IPD?  To  answer  this  question,  we  turn  to  the  estimates  scaling  feature 
within  JMP.  Figure  16  displays  the  scaled  coefficient  estimates  of  the  linear 
regression  model. 


Scaled  Estimates 


Continuous  factors  centered  by  mean,  scaled  by  range/2 

Term 


Scaled 

Estimate 


Intercept 

NumRed 

NumBlue 

BlueToRed 

NumSensors 

ProbDelay 

AccVar 

(NumRed-1 7. 1 392)*(NumBlue-25.7089) 

( NumRed- 17.1 392)*(NumSensors-5. 12121) 

( NumRed- 17.1 392)*(  ProbDelay-0.40242) 

( NumRed- 17.1 392)*(AccVar-3. 1 71 72) 

(NumBlue-25.7089)*(BlueToRed-1.5) 

(NumBlue-25.7089)*(NumSensors-5.12121) 

(NumBlue-25.7089)*(ProbDelay-0.40242) 

(NumBlue-25.7089)*(AccVar-3. 1 71 72) 

(BlueToRed-1.5)*(NumSensors-5.12121) 

( BlueToRed- 1 .5)*(  ProbDelay-0.40242) 

( BlueToRed- 1.5)*(AccVar-3. 171 72) 
(NumBlue-25.7089)*(BlueToRed-1.5)*(NumSensors-5. 12121) 
(NumBlue-25.7089)*(BlueToRed-1.5)*(ProbDelay-0.40242) 
(NumBlue-25.7089)*(BlueToRed-1.5)*(AccVar-3.17172) 
(NumRed-17.1392)*(NumRed-17.1392) 

( NumRed- 17.1 392)*(NumRed- 17.1 392)*(NumRed- 17.1 392) 

(NumBlue-25.7089)*(NumBlue-25.7089) 

(NumBlue-25.7089)*(NumBlue-25.7089)*(NumBlue-25.7089) 

( BlueToRed- 1 .5)*(  BlueToRed- 1 .5) 

( BlueToRed- 1 .5)*(  BlueToRed- 1 ,5)*(  BlueToRed- 1 .5) 

(NumSensors-5.12121)*(NumSensors-5.12121) 

(NumSensors-5.12121)*(NumSensors-5.12121)*(NumSensors-5.12121) 


0.3474839 

-0.492861 

0.9988335 

-0.735485 

0.0184305 

-0.038141 

-0.056029 

-0.679629 

-0.032452 

0.0214634 

0.0788608 

-1.026712 

0.0804325 

-0.063958 

-0.068975 

-0.070761 

0.056808 

0.0471198 

-0.073513 

0.0641396 

0.0572128 

0.0317803 

0.0918711 

0.8671578 

-0.092617 

-0.008405 

0.0533228 

-0.035649 

0.051468 


Std  Error 

0.000226 

0.004289 

0.008905 

0.006131 

0.000423 

0.000228 

0.000242 

0.006594 

0.000659 

0.000667 

0.000698 

0.00895 

0.001351 

0.001368 

0.001432 

0.00097 

0.000982 

0.001028 

0.000697 

0.000705 

0.000738 

0.001275 

0.000731 

0.008121 

0.001645 

0.00236 

0.000966 

0.000278 

0.000537 


t  Ratio 

1534.36 

-114.92 

112.17 

-119.96 

43.59 

-167.49 

-231.89 

-103.07 

-49.26 

32.19 
112.97 

-114.72 

59.51 
-46.77 
-48.17 
-72.93 

57.86 
45.83 

-105.52 

90.98 

77.51 
24.92 

125.66 

106.78 

-56.31 

-3.56 

55.20 
-128.40 

95.87 


Prob>|t| 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

<0001* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

0.0000* 

<0001* 

0.0000* 

0.0000* 

0.0000* 

0.0004* 

0.0000* 

0.0000* 

0.0000* 


Figure  16.  Scaled  Estimates  of  the  Linear  Regression  Coefficients 


This  graphic  displays  in  which  direction  (positive  or  negative),  and  to  what 
degree,  a  particular  parameter  coefficient  is  having  on  the  response,  which,  in  this  case,  is 
the  IPD.  This  feature  was  actually  used  to  eliminate  about  12  other  parameters  that  were 
originally  included  by  the  JMP  software.  However,  removing  these  12  parameters 
dropped  the  R-square  by  only  0.00261.  Examining  Figure  16,  one  might  argue  that  even 
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more  parameters  could  be  removed.  Nonetheless,  the  figure’s  message  clearly  comes 
through  that  force  sizes  and  force  ratios  are  far  and  away  the  most  important  factors.  In 
fact,  taking  the  top  six  scaled  estimates  (number  of  red,  number  of  blue,  blue-to-red  ratio, 
blue  *  blue-to-red  ratio,  number  of  red  ,  and  number  of  blue")  to  build  a  regression  would 
yield  an  R-square  of  0.71.  Noting  the  diminished  impact  of  the  sensors  on  the  outcome, 
we  still  want  to  measure  the  degree  of  its  impact,  broken  down  by  sensor,  if  possible.  To 
do  this,  we  will  turn  to  some  visual  aids  to  assist  in  this  endeavor. 

C.  IDENTIFYING  GRAPHICAL  TRENDS 

With  only  five  model  parameters,  we  have  more  flexibility  to  use  traditional 
graphs  and  plots  to  aid  in  visualizing  the  data.  Figure  17  is  a  very  simple  summary  of  the 
results  showing  the  red  score  for  a  given  number  of  red  and  blue  averaged  across  all  the 
variations  of  the  other  three  parameters.  These  new  data  have  been  overlaid  on  top  of  the 
data  from  Figure  14  to  show  the  improvement  of  the  more  complicated  game  over  the 
expected  score  for  a  simple  TPZS  game.  Recall  that  a  TPZS  game  is  without  sensors, 
and  the  more  complicated  game  involves  multiple  sensors  and  takes  into  account 
information  quality.  Observe  that  for  all  instances  where  the  number  of  red  units  is 
greater  than  five,  blue  dominates  red  with  the  use  of  sensors  in  terms  of  improving  over 
the  previous  expected  value  of  a  TPZS  game.  Even  in  cases  where  red  has  less  than  five 
units,  the  blue  side  still  does  well  in  many  instances,  although  note  that  the  margin  of 
improvement  is  noticeably  diminished.  Also  note  that  the  improvement  is  not  uniform,  in 
which  case  the  lines  would  be  parallel.  In  fact,  we  observe  that  when  blue  is  at  its  least 
amount,  the  improvements  are  slim;  they  steeply  increase  until  a  point  where  the 
increased  improvement  with  the  increase  in  blue  units  becomes  more  modest.  Finally, 
another  turning  point  is  reached  where  the  degree  of  improvement  begins  decreasing 
again  until  blue  is  at  its  maximum  number;  the  increase  from  sensors  is  practically 
nothing.  This  suggests  the  varying  importance  of  information  while  supporting  the 
observation  in  the  previous  section  concerning  the  importance  of  force  size  and  force 
ratio. 
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Comparison  of  Red  Scores  with  Sensors  to  Red  Scores  without  Sensors 
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Figure  18  characterizes  the  percentage  difference  in  red  scores  shown  in  Figure 
17.  Figure  18  shows  much  more  clearly  the  steep  increase  in  the  improved  percent 
decrease  initially  enjoyed  by  blue  and  then  the  plateau  at  the  top,  where  the 
improvements  are  more  modest.  Finally,  the  drop-off  is  as  dramatic  as  the  improvement. 
Also  noted  on  the  figure  are  the  estimated  values  of  blue  associated  with  these  plateaus. 
Once  the  number  of  red  units  increases  to  13,  the  magic  number  for  blue  seems  to  be  10, 
at  which  point,  blue  enjoys  a  significant  degradation  to  the  red  score  until  the  number  of 
blue  units  reaches  about  one  and  half  times  that  of  the  number  of  red  units.  This  again 
reinforces  the  force  size  observation,  but  it  also  begins  to  quantify  the  degree  to  which 
red  is  being  degraded.  Observe  that  for  the  peak  cases,  blue  is  reducing  red  anywhere 
from  22%  to  33%.  However,  for  most  of  the  cases,  blue  is  reducing  the  red  score  by  less 
than  20%,  particularly  in  the  situation  where  red  has  less  than  13  units.  This  also  partly 
explains  why  the  number  of  sensors  was  shown  to  have  a  minimal  impact  in  the 
regression  model. 
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Figure  18. 


Summary  of  the  Improved  %  Decrease  in  Over  the  Range  of  Blue  Units  for  a  Given  Number  of  Red  Units 
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D.  VALUE  OF  VARIOUS  LEVELS  OF  SENSORS 


The  sensor  parameter  was  varied  from  one  sensor  up  to  nine  sensors.  Blue  having 
nine  sensors  means  that  blue  had  a  sensor  located  at  every  possible  red  location.  This 
section  will  examine  the  various  performances  of  the  nine  different  levels  independently. 

Figure  19  displays  contour  plots  of  the  data  for  each  sensor,  while  varying  the 
blue-to-red  ratio  and  the  number  of  blue  units,  which  we  already  know  are  very  important 
to  the  improved  percent  decrease  to  red.  The  contour  plots  are  useful  for  pointing  out 
critical  areas.  They  are  similar  to  viewing  a  topographic  map,  where  lines  that  are  close 
together  represent  a  rapid  change  in  terrain  and  lines  further  apart  represent  more  gradual 
change  or  plateaus.  We  are  looking  for  clear  patterns  related  to  in  the  levels  for  each 
sensor,  as  well  as  differences  among  the  sensors.  Note  that  the  contour  plots  of  Figure  19 
(on  the  following  page)  are  scaled  uniformly  for  ease  of  comparison. 

The  first  thing  to  note  is  the  similarity  between  the  locations  of  the  plateau  on  all 
the  plots.  All  of  the  plateaus  rest  roughly  between  a  blue-to-red  ratio  of  two-thirds  to  two 
and  between  10  blue  units  and  40  blue  units.  This  coincides  with  our  observation  in 
Section  C  concerning  the  range  where  blue  gets  maximum  value  from  its  sensors.  The 
second  thing  to  note  is  the  difference  in  the  plots.  They  all  have  roughly  the  same  shape, 
but  a  significant  color  change  occurs  between  one  and  two  sensors,  indicating  a 
significant  jump  in  perfonnance  between  these  two  levels.  We  observe  a  slight  change 
between  two  and  three  sensors,  three  and  four,  and  four  and  five.  However,  once  we 
reach  five  sensors,  there  is  hardly  any  noticeable  change  between  any  additional  sensors 
after  five.  This  suggests  that  we  have  reached  the  maximum  potential  of  our  information 
advantage  at  five  sensors.  Thus,  five  sensors  appear  to  do  as  well  as  the  scenario  where 
there  is  a  sensor  at  every  location.  We  will  break  this  out  further  to  capture  the  value  of 
these  sensors  in  terms  of  the  number  of  blue  units. 
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Figure  19. 


Contour  Plots  by  Sensor  for  Number  of  Blue  Units  vs.  Blue  to  Red  Ratio 
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Given  the  information  above,  we  can  get  another  perspective  with  a  little  more 
detail  on  the  impact  of  varying  sensor  levels  using  the  line  graph  methodology  employed 
previously.  Figures  20  and  21  are  very  similar  to  Figure  18  in  terms  of  shape,  except  for 
two  things:  1)  each  level  of  sensor  is  shown  independently  and  2)  the  y-axes  are  different 
in  the  case  of  Figure  21.  Figure  20  is  somewhat  of  an  addition  to  Figure  18,  showing  the 
breakout  of  the  value  of  each  sensor  in  terms  of  the  improved  percent  reduction  to  red 
score.  For  Figure  21,  the  blue  unit  value  on  the  y-axis  was  calculated  by  determining  the 
number  of  blue  units  required  to  achieve  a  similar  expected  value  to  that  achieved  by  the 
use  of  the  respective  number  of  sensors. 

The  first  thing  to  note  is  that  the  size  of  the  jump  in  performance  between  the 
various  sensor  levels  is  readily  apparent  in  both  figures.  As  with  the  contour  plots,  we 
see  a  sizeable  jump  in  performance  even  with  only  one  sensor.  Additionally,  we  can 
generally  observe  that  the  results  are  fairly  consistent  for  two,  three,  or  four  sensors — at 
least  until  the  number  of  red  units  is  equal  to  20.  Also,  while  there  may  be  a  statistical 
difference  between  five  or  more  sensors,  there  is  no  practical  importance  when  a 
conservative  estimate  for  that  difference  is  something  less  than  a  2%  improved  decrease. 
All  of  these  observations  conform  to  those  of  the  contour  plots  as  shown  previously, 
although  a  little  bit  more  variation  is  visible  with  the  detailed  breakout.  More 
importantly,  however,  is  the  relative  comparison  that  can  be  made  between  the  number  of 
sensors  and  the  number  of  blue  units.  Figure  20  should  be  read  by  considering  a  situation 
where  there  are  12  red  units  and  10  blue  units.  In  this  scenario,  if  we  want  to  determine 
the  relative  values  of  sensors,  we  find  the  set  of  data  respective  to  12  red  units,  identify 
the  approximate  location  of  10  blue  units,  and  read  the  values  from  the  y-axis  for  each 
sensor  value.  In  this  case,  1  sensor  is  equal  to  about  six  blue  units;  2,  3,  and  4  sensors 
between  7  and  9  blue  units;  and  5  or  more  sensors  about  12  blue  units.  Within  this 
abstraction,  it  is  difficult  to  detennine  whether  one  sensor  is  worth  six  blue  units,  but 
these  are  the  types  of  comparisons  we  want  to  be  able  to  draw  when  considering  real 
equipment.  However,  we  once  again  observe  that  costs  are  not  unifonn  across  the  board. 
The  conditions  of  this  model  suggest  that  a  good  strategy  involves  building  force  size 
initially,  adding  some  sensors,  and  then  adding  more  force  size  which  equates  to  a 
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nonlinear  function  of  blue  units  and  blue  sensors  mixes.  Table  6  shows  various  side-by- 
side  slices  taken  out  of  Figures  20  and  21  when  the  number  of  red  is  equal  to  17  for 
various  numbers  of  blue  to  show  these  ideas  more  clearly.  The  equivalent  blue  is  equal  to 
the  increase  in  the  number  of  blue  required  to  achieve  the  same  percent  improved 
decrease  that  is  listed  above  it.  Thus,  in  the  case  of  where  blue  has  10  units  and  1  sensor 
the  percent  decrease  is  14%.  If  blue  wanted  to  have  a  similar  effect  purely  by  increasing 
the  number  of  blue  he  would  need  an  additional  7.2  blue  added  to  his  current  number  of 
17.  Moving  across  a  row,  notice  how  the  percent  improved  increase  goes  up  until  the  last 
column,  where  in  all  cases  the  value  goes  down.  Also  notice  that  as  we  move  down  a 
column  there  is  a  noticeable  jump  in  value  from  1  to  2  sensors  but  everything  tops  out  at 
5  sensors  at  which  point  no  large  increases  are  made. 


Number  of  Red  =  17 

Number  of  Blue 

Sensors 

Value  Type 

6 

10 

17 

27 

40 

1 

%  Imp  Decrease 

9.8% 

13.3% 

14.0% 

19.2% 

15.7% 

Equivalent  Blue 

5.0 

6.8 

7.2 

9.8 

8.0 

2 

%  Imp  Decrease 

18.5% 

23.2% 

25.8% 

31.8% 

19.0% 

Equivalent  Blue 

9.5 

11.8 

13.1 

16.2 

9.7 

3 

%  Imp  Decrease 

19.0% 

26.2% 

26.6% 

30.5% 

18.9% 

Equivalent  Blue 

9.7 

13.3 

13.6 

15.6 

9.6 

4 

%  Imp  Decrease 

20.0% 

27.5% 

29.5% 

32.6% 

19.5% 

Equivalent  Blue 

10.2 

14.0 

15.0 

16.6 

10.0 

5 

%  Imp  Decrease 

22.0% 

28.9% 

32.2% 

36.7% 

20.5% 

Equivalent  Blue 

11.2 

14.8 

16.4 

18.7 

10.4 

6 

%  Imp  Decrease 

21.5% 

27.7% 

31.0% 

37.0% 

20.7% 

Equivalent  Blue 

11.0 

14.1 

15.8 

18.9 

10.5 

7 

%  Imp  Decrease 

21.7% 

27.8% 

31.2% 

36.5% 

20.4% 

Equivalent  Blue 

11.1 

14.2 

15.9 

18.6 

10.4 

8 

%  Imp  Decrease 

21,7% 

29.0% 

33.1% 

37.8% 

20.6% 

Equivalent  Blue 

11.1 

14.8 

16.9 

19.3 

10.5 

9 

%  Imp  Decrease 

21.7% 

28.7% 

32.6% 

36.7% 

20.5% 

Equivalent  Blue 

11.1 

14.6 

16.6 

18.7 

10.5 

Table  6.  Comparing  Sensor  Performance  with  Equivalent  Number  of  Blue  Units 
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Figure  20.  Improved  Percent  Decrease  in  Red  Score  by  Sensor  per  Number  of  Red  and  Number  of  Blue 
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Figure  2 1 .  Sensor  Value  in  Terms  of  Blue  Units  by  Sensor  per  Number  of  Red  and  Number  of  Blue 
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E.  ASSESSING  RISK  AND  THE  IMPACT  OF  INFORMATION  QUALITY 

One  of  our  immediate  intuitions  concerning  the  heavy  use  of  information  is  that 
there  will  be  times  when  performance  will  drop  below  a  certain  threshold  that  would  not 
have  been  possible  otherwise,  since  we  leave  ourselves  open  to  deception  or  simple 
human  errors  on  the  friendly  side.  While  the  results  show  a  general  improvement  to  the 
expected  value,  this  only  represents  the  expected  average — it  does  not  take  into  account  a 
worst-case  scenario.  Within  the  context  of  the  TPZS  game,  if  blue  plays  his  optimal 
strategy,  he  can  guarantee  a  maximum  red  score.  This  maximum  red  score  can  be 
calculated  by  determining  the  minimal  number  of  blue  units  possible  at  any  one  of  the 
three  objectives.  In  trying  to  establish  information  superiority,  it  is  possible  for  blue  to 
deviate  from  his  original  strategy,  thereby  giving  opportunity  for  the  red  side  to  surpass 
this  minimum  number.  This  is  the  inherent  risk  in  pursuing  information  superiority.  For 
this  experiment,  we  can  calculate  the  number  of  instances  when  this  occurred. 

Conversely,  in  light  of  this  risk,  there  is  also  the  potential  for  increased  gain.  It  is 
in  realizing  this  potential  gain  that  blue  is  able  to  improve  upon  the  original  expected 
value.  Again,  going  back  to  the  TPZS  game,  in  addition  to  a  minimum  red  score,  there  is 
also  a  maximum  decrease  that  is  possible  when  blue  is  using  his  optimal  strategy.  Unless 
blue  is  able  to  achieve  some  other  advantage,  blue  will  never  be  able  to  improve  upon 
this  value.  Yet,  this  is  what  an  information  advantage  is  able  to  do.  It  allows  blue  to 
capitalize  on  degrading  the  red  capability. 

Figure  22  captures  this  infonnation  succinctly  by  segmenting  the  data  by  the 
number  of  sensors,  and  then  showing  how  many  data  sample  scores  were  below  the 
minimum  and  how  many  beat  the  maximum.  Overall,  we  observe  that  about  2%  of  the 
sample  proportion  was  below  the  minimum,  while  about  85%  of  the  proportion  was 
above.  That  alone  is  evidence  that  while  there  is  risk,  the  gains  appear  worth  the  risk. 
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Frequencies 


Level 

Count 

Prob 

1 

284400 

0.06061 

2 

568800 

0.12121 

3 

568800 

0.12121 

4 

568800 

0.12121 

5 

568800 

0.12121 

6 

568800 

0.12121 

7 

568800 

0.12121 

8 

568800 

0.12121 

9 

426600 

0.09091 

Total 

4692600 

1.00000 

N  Missing 

0 

9  Levels 


Total  Number  by  Sensor 


Frequencies 


Level 

Count 

Prob 

1 

5524 

0.05396 

2 

17355 

0.16954 

3 

21985 

0.21477 

4 

19543 

0.19092 

5 

7860 

0.07679 

6 

7831 

0.07650 

7 

7620 

0.07444 

8 

8128 

0.07940 

9 

6517 

0.06367 

Total 

102363 

1.00000 

N  Missing 

0 

9  Levels 


Total  Number  Below 


Frequencies 


Level 

Count 

Prob 

1 

218304 

0.05451 

2 

475973 

0.11884 

3 

470587 

0.11750 

4 

474533 

0.11849 

5 

497323 

0.12418 

6 

498258 

0.12441 

7 

498251 

0.12441 

8 

499180 

0.12464 

9 

372591 

0.09303 

Total 

4005000 

1.00000 

N  Missing 

0 

9  Levels 


Total  Number  Above 


Figure  22.  Total  Samples  by  Sensor,  Samples  Below  Minimum,  Samples  Above 

Maximum 

Table  7  summarizes  the  data  from  Figure  22  into  a  comparative.  From  it  we  see 
there  appears  to  be  no  particular  advantage  in  terms  of  the  number  of  sensors  in 
considering  risk  or  gain. 


NumSensors 

%  Below 
Min 

%  Above 
Max 

1 

1.94% 

76.76% 

2 

3.05% 

83.68% 

3 

3.87% 

82.73% 

4 

3.44% 

83.43% 

5 

1.38% 

86.73% 

6 

1.38% 

87.60% 

7 

1.34% 

87.60% 

8 

1.43% 

87.76% 

9 

1.53% 

87.34% 

Table  7.  Comparison  of  Risk  vs.  Gain  by  Sensor 
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In  considering  the  effect  of  information  quality  on  the  results,  we  again  turn  to 
contour  plots.  Figure  22  plots  the  probability  of  time  delay  by  the  accuracy  variation 
parameter  for  each  particular  sensor.  First,  note  that  our  intuition  is  confirmed — 
decreasing  information  quality  results  in  a  decreased  effect  of  our  sensors.  In  this  case, 
as  we  increase  the  probability  of  time  delay  and  the  accuracy  variation,  the  quality  of  our 
results  decrease.  Interestingly,  the  slope  remains  fairly  steady  and  constant  for  all  levels 
of  sensors.  There  appear  to  be  no  points  where  once  the  information  gets  so  bad,  there  is 
a  significant  drop  in  the  improved  percent  decrease.  The  plots  also  suggest  that 
increasing  the  number  of  sensors  can  overcome  some  of  the  effects  of  degraded 
information.  However,  recall  the  observations  from  the  scaled  estimates  of  the  linear 
regression  model  indicate  that,  while  probability  of  time  delay  and  accuracy  variation  are 
included  in  the  model,  their  impact  is  limited.  It  is  possible  that  these  two  parameters  are 
simply  being  overrun  by  the  other  information  in  the  model.  Thus,  it  appears  we  cannot 
draw  any  strong  conclusion  from  the  contour  plots.  Chapter  V  will  go  on  to  summarize 
the  major  findings  and  discuss  potential  future  research  areas. 
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Figure  23.  Contour  Plots  by  Sensor  of  Probability  of  Delay  vs.  Accuracy  Variation 
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V.  CONCLUSIONS 


A.  SUMMARY  OF  FINDINGS 

For  the  given  circumstances  of  this  model,  the  data  have  shown  that  the  value  of 
information  superiority  is  not  uniform,  but  is  strongly  influenced  by  force  ratio  and  force 
size.  This  does  not  suggest  that  information  superiority  has  no  value;  on  the  contrary,  we 
measured  the  added  value  in  this  study.  However,  it  does  suggest  the  possibility  that  our 
zeal  for  information  superiority  should  be  tempered  by  the  fact  that  force  size  plays  a 
significant  role  as  well.  There  are  “knees  in  the  curve”  at  both  ends  of  the  force  size 
spectrum.  First,  force  size  must  be  increased  to  a  certain  level  before  information 
superiority  can  be  utilized  to  its  maximum  extent.  Second,  once  a  certain  force  level  is 
reached,  information  superiority  begins  to  decrease  in  value.  This  is  another  reminder  of 
the  importance  of  striking  the  balance  between  sensor  and  shooter  and  why  this  area  of 
research  is  so  important.  Additionally,  this  relationship  suggests  that  information 
capabilities  should  not  be  viewed  as  a  simple  add-on  to  force  capability,  but  that  the 
values  of  force  size  and  information  are  dependent  on  one  another.  Thus,  force 
development  must  incorporate  and  evaluate  the  combined  capabilities  of  information 
systems  and  combat  equipment,  and  not  assess  these  capabilities  individually. 

This  study  also  suggests  that  information  superiority  can  be  increased  to  a  point 
after  which  additional  information  superiority  translates  into  little  or  no  value  in  decision 
superiority  on  the  battlefield.  A  chart  of  the  general  value  per  sensor  appears  in  Figure 
23. 


55 


Number  of  Sensors 


Figure  24.  General  Value  of  Information  Superiority  by  Sensor 


Specifically,  we  see  that  increasing  the  number  of  sensors  beyond  five,  provides 
little  to  no  added  value.  We  saw  that  while  there  may  be  a  statistical  significance,  the 
additional  sensors  are  of  no  practical  value.  Flowever,  there  are  a  couple  of  critical 
junctures.  For  example,  the  addition  of  a  single  sensor  provides  a  noticeable  percent 
decrease  in  the  red  score,  suggesting  that  even  a  relatively  small  rise  in  information 
superiority  can  have  an  immediate  impact.  Adding  an  additional  sensor — bringing  the 
total  to  two — again  provides  a  recognizable  increase  to  blue  performance.  The  addition 
of  three  and  four  sensors  appears  to  have  little  impact,  with  the  next  jump  in  performance 
occurring  at  five  sensors  and  going  no  higher.  To  some  degree,  this  is  counterintuitive. 
Given  a  situation  where  blue  has  nine  sensors  and  every  location  covered,  we  would  tend 
to  believe  that  this  scenario  would  dominate  all  others.  This  is  not  the  case,  however,  and 
it  poses  some  interesting  questions  to  those  who  are  interested  in  developing  persistent 
intelligence,  surveillance,  and  reconnaissance  (ISR)  systems  and  the  role  they  should 
play.  While  the  work  required  by  blue  in  executing  its  decision  was  not  specifically 
measured,  it  should  be  clear  that  an  increased  workload  was  applied  to  the  blue  force 
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when  there  were  nine  sensors  (vice  only  five)  due  to  the  increased  calculation  required. 
However,  there  was  little  to  no  measurable  resulting  improvement.  This  indicates 
another  potential  area  of  cost  savings  in  terms  of  the  tax  placed  on  the  decision  maker, 
and  reinforces  the  importance  of  identifying  the  proper  scope  for  information  superiority. 

Finally,  the  data  reveal  no  critical  points  in  terms  of  infonnation  quality,  but 
rather,  the  slope  of  the  decrease  in  performance  as  a  result  of  increasingly  poor 
information  remains  steady.  The  data  suggest  a  slight  mitigation  of  poor  information  by 
the  increase  in  the  number  of  sensors,  but  probably  not  to  the  degree  sufficient  to  justify 
the  addition  of  sensors.  Additionally,  while  the  data  show  that  the  gains  of  information 
superiority  far  outweigh  the  risks,  there  are,  nonetheless,  risks  that  need  to  be  considered. 
No  differences  in  risk  or  gain  are  noted  to  any  noticeable  degree  based  on  the  number  of 
sensors. 

B.  RECOMMENDATIONS 

There  are  a  number  of  ways  in  which  to  extend  this  research  into  even  more 
profitable  studies.  We  will  discuss  a  few  of  them  here. 

First,  without  changing  the  current  model  there  is  room  for  additional  exploration 
by  adjusting  the  current  design  of  experiment.  In  this  experiment,  a  large  proportion  of 
the  design  points  were  used,  ensuring  that  all  possible  combinations  of  red  units  and  blue 
units  were  considered.  This  was  at  the  expense  of  considering  an  increasing  number  of 
variations  of  sensor  number,  probability  of  time  delay,  and  accuracy  variation.  Recall 
that  there  were  948  combinations  of  red  and  blue  and  an  NOLH  design  that  totaled  99 
rows.  This  represents  somewhat  of  a  disparity,  particularly  if  one  is  interested  more 
keenly  in  the  impact  of  infonnation  quality.  To  do  this,  we  recommend  that  red-blue 
combinations  be  limited  to  those  plateau  regions  identified  in  the  contour  plots  and  an 
increased  number  of  design  points  be  created  for  the  other  three  parameters.  Not  only 
would  this  provide  more  substance  to  the  findings  of  this  study,  but  it  would  also  allow 
for  a  more  detailed  analysis  of  the  risk  versus  gain  issue.  A  potential  question  to  be 
answered  is,  “How  sensitive  are  risk  and  gain  to  infonnation  quality?”  In  other  words,  is 
risk  or  gain  substantially  increased  or  decreased  at  some  level  of  information  quality? 
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Furthermore,  information  superiority  could  be  explored  from  the  red  perspective,  where 
red  receives  the  infonnation  advantage.  Consideration  could  even  be  given  to  the 
circumstance  where  both  sides  are  provided  varying  levels  of  sensors  at  the  same  time. 
All  of  these  would  provide  added  value  to  this  research. 

There  are  also  a  number  of  ways  to  extend  the  model  itself.  Worthy  of 
exploration  would  be  the  examination  of  varying  the  scoring  system  at  the  objectives. 
This  could  be  accomplished  by  changing  the  basic  game  to  some  variation  of  a  Blotto 
game,  for  example.  A  Blotto  game  is  a  type  of  two-person  zero  sum  game  that  can  still 
be  solved  using  linear  programming  techniques.  An  advantage  to  Blotto  games  is  that 
they  have  been  broadly  studied,  thus  providing  additional  background  to  the  research. 
However,  major  adjustments  would  have  to  be  made  to  the  current  implementation  of  the 
model  to  take  into  account  the  different  optimal  strategies  that  would  be  employed  by 
both  red  and  blue.  Additionally,  it  is  highly  likely  that  the  decision  algorithm  employed 
by  blue  would  appreciably  increase  in  complexity.  Identifying  an  appropriate  decision 
algorithm,  even  in  a  relatively  simple  model  such  as  the  one  represented  here,  can  be  a 
challenging  task.  However,  the  value  of  the  study  would  be  in  examining  the  value  of 
information  superiority  over  a  broad  range  of  decision-making  scenarios  and  looking  for 
broader  trends  in  the  value  of  information  superiority  to  decision  making. 

A  variety  of  other  complexities  could  be  incorporated  that  would  also  require 
extensive  model  development.  Consideration  could  be  given  to  implementing  types  of 
information  with  varying  degrees  of  importance.  This  would  require  a  potential 
weighting  scheme  with  each  of  these  types  of  information.  Additionally,  the 
incorporation  of  non-homogenous  red  and  blue  forces  goes  hand  in  hand  with  adding 
types  of  information.  A  simpler  means  to  increase  complexity  would  be  to  expand  the 
size  of  the  gameboard  and  potentially  adjust  the  current  movement  restrictions  on  both 
red  and  blue.  Adjusting  the  levels  of  movement  restrictions  is  akin  to  varying  the  level  of 
flexibility  inherent  to  the  decision  maker.  The  relationship  between  flexibility  and  the 
value  of  information  superiority  would  be  an  interesting  area  of  study  itself. 
Furthermore,  implementing  a  method  for  different  types  of  sensors  with  probabilities  of 
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detection  that  differ  by  type  of  sensor  would  represent  a  huge  leap  in  complexity.  This 
would  be  like  a  weapons-target  assignment  problem  but  with  sensors,  where  questions  of 
sensor  mixes  and  sensitivity  to  probability  of  detection  could  be  addressed. 

These  suggestions  represent  a  slice  of  the  many  ways  this  research  can  be 
improved  and  extended.  Game  theory  is  the  common  thread  and  represents  a  rich  and 
robust  subject  area  that  has  a  strong  history  of  addressing  the  operational  issues  facing 
today’s  decision  makers.  While  not  revolutionizing  the  study  of  information  superiority, 
this  research  demonstrated  that  game  theory  can  be  effectively  applied  in  gamering 
insights  into  a  critical  issue  facing  the  military  operations  research  community.  As  a 
community  we  must  be  able  to  effectively  tackle  the  role  and  value  of  infonnation  on  the 
battlefield.  Future  research  should  continue  to  exploit  game  theory  concepts  and 
applications  to  further  explore  the  questions  posed  by  this  study  and  related  questions. 
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